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Abstract 

A  performance  management  team^was  implemented  at  the 
Engineering  Branch  of  the  2750  Civil  Engineering  Squadron  at 
Wr i ght-Patterson  Air  Force  Base.  The  team  focused  on  the 
process  of  reviewing  projects  designed  by  architect- 

engineers  and  the  Corps  of  Engineers.  As  a  result  of  this 

/  <n o.*' 

team  ef  f  ort ,  '-improvements  to  the  process  were  made^.  More 

timely  and  better  comments  were  being  provided  to  the 
designers,  af ter  the  improvement  actions. 

From  the  experiences  gained  in  the  analysis  of  this 
case ,  three  day  training  course  was  developed.  This 
course  was  created  to  provide  the  tools  and  attitudes  needed 
to  begin  performance  management  efforts  in  other  Engineering 
Branches.  The  course  was  designed  to  provide  a  starting 
point,  and  as  a  guide  to  the  performance  management  effort. 

The  course  ^was  structured  to  be  presented  at  the 
squadron  location,  in  order  to  teach  people  in  their  own 
environment.  Topics  included  were  the  distinctives  of 
the  engineering  environment,  understanding  the  process 
framework,  the  use  of  measurement  in  evaluating  work 
processes,  and  employee  participation.  The  use  of *the 
course  was  recommended  for  the  base  level  Engineering 
Branch,  t  ■'  •  >'  "A  S  ■  f'\o  r  a  •"  t  f'  A''? .  ..?,■<  '  ~~  A  -  -  *  $\ 
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A  PERFORMANCE  MANAGEMENT  FRAMEWORK 
FOR  CIVIL  ENGINEERING 


I .  Introduction 


Genera  1^  Issue 

The  Department  of  Defense  has  recognized  the  potential 
for  benefits  from  the  implementation  of  Total  Quality 
Management  (TQM)  concepts  in  military  organizations.  To 
achieve  these  benefits  the  Secretary  of  Defense  has 
instructed  military  organizations  to  implement  TQM 
(Department,  1989b:2).  Air  Force  Civil  Engineering 
Squadrons  are  among  the  organizations  affected  by  this 
instruction.  Direction  is  needed  regarding  how  to 
accomplish  this  implementation  in  a  Civil  Engineering 
Squadron,  as  well  as  guidance  in  the  type  of  benefits  to  be 
sought . 

TQM  is  gaining  great  publicity  in  organizations 
throughout  the  country.  Most  of  this  exposure,  however,  is 
in  manufacturing.  Service  organizations,  such  as  a  Civil 
Engineering  Squadron,  have  needs  and  goals  which  differ  from 
their  manufacturing  counterparts.  Likewise,  application  of 
TQM  in  military  and  government  organizations  will  encounter 
different  obstacles  than  in  private  organizations  where 
profit  is  the  goal.  To  be  used  successfully,  the  TQM 
concepts  need  to  be  adapted  to  the  unique  nature  of  an  Air 
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Force  Civil  Engineering  Squadron.  To  differentiate  the 
management  philosophy  needed  for  civil  engineering  from  TQM, 
the  term  performance  management  will  be  used  to  describe  the 
attitudes,  tools,  and  framework  needed  to  direct  improvement 
efforts  in  this  setting. 


Government  and  industry  have  come  to  understand 
that  previously  acceptable  norms  of  goods  and 
services  are  no  longer  acceptable.  Customer 
satisfaction,  reliability,  productivity,  costs,  and 
for  industry,  market  share,  profitability,  and  even 
survival  are  directly  affected  by  the  quality  of  an 
organization’s  products  and  performance. 

Therefore,  it  becomes  essential  to  develop 
attitudes  and  systems  --  at  all  levels  of  an 
organization  --  that  promote  and  implement 
continuous  improvement  of  procedures,  processes, 
products,  and  services.  Those  attitudes  and  systems 
are  the  focus  of  Total  Quality  Management  (TQM). 
(Department,  1989a: 2) 

The  drive  for  performance  management  can  be  summed  up 
in  the  old  adage,  *  an  ounce  of  prevention  is  worth  a  pound 
of  cure.’  For  too  long  managers  both  within  and  outside  of 
government  have  been  content  to  let  well  enough  alone  and 
only  act  when  problems  became  apparent.  A  performance 
management  framework  must  emphasize  the  importance  of 
analyzing  the  status  of  performance  today  and  seeking  to 
improve  it  for  tomorrow  (Department,  1989a:8). 

The  kind  of  perspective  management  needs  is  one  which 
is  future  oriented.  Quality  provides  just  such  a 
perspective.  Quality  is  an  important  ingredient  in  overall 
performance.  When  managers  make  the  quality  of  their 
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products  and  processes  the  target  of  their  efforts,  they  are 
looking  toward  being  competitive  in  the  future  as  well  as 
today  (Townsend  and  Qebhardt ,  1989:20).  Quality  products 

are  products  which  endure,  ensuring  that  there  will  be 
demand  for  such  products  long  after  other  competitors  have 
disappeared . 


Figure  1.  Philosophy  Change  for  Performance  Management 

Performance  improvement  is  an  ongoing  process.  There 
is  never  a  point  where  the  concerned  manager  can 
realistically  say  'this  is  as  good  as  it  can  get."  At  any 
given  time,  there  will  always  be  improvements  which  can  yet 
be  made.  Managers  need  to  make  a  commitment  to  scrutinize 
the  activity  of  organizations  continuously  to  determine 
where  additional  improvements  might  be  made.  When  one  of 
these  improvements  is  implemented,  attention  needs  to  be 
given  to  other  aspects  of  performance  which  can  also  be  made 
better.  (Roth,  1989:26) 
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Among  profit-making  companies,  it  has  been  declared 
that  quality  is  the  proposal  which  affects  the  profits  of  a 
company  most.  In  fact,  it  has  been  estimated  that  the  cost 
to  American  companies  due  to  inferior  quality  is  in  excess 
of  thirty  percent  of  their  gross  sales  (Townsend  and 
Gebhardt,  1989:18).  This  alarming  figure  is  a  direct 
challenge  to  improve  quality.  The  government  organization 
should  not  consider  itself  an  exception.  While  not  having 
profit  as  a  measure  for  quality  of  performance,  untold 
resources  are  squandered  due  to  poor  quality.  Thi3  waste 
results  from  rework  after  poor  initial  work,  and  from 
inefficiency  of  operation. 

Performance  management  has  three  principal  orientations 
by  which  organizations  are  encouraged  to  pursue  quality: 
llcustomer  orientation;  2)process  orientation; 

3) participation  orientation.  First,  the  customer 
orientation  forces  identification  of  the  internal  and 
external  customers  of  the  organization.  Satisfaction  of 
these  customers  then  becomes  the  measure  of  quality  in  this 
orientation  (Department,  1989a:6-7).  Internal  customers  are 
those  within  the  organization  who  receive  output  from  the 
particular  unit.  For  example,  when  the  engineering 
department  prepares  a  drawing  showing  the  maintenance 
department  how  to  install  a  new  component,  the  maintenance 
personnel  are  internal  customers  of  the  engineering 
department.  External  customers  are  those  outside  the 
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organization  which  receive  output  or  products.  The 
organization  then  sets  better  service  to  these  customers  as 


a  goal  (Kacker ,  1988:40).  For  an  engineering  company  in 

particular,  the  degree  to  which  it  meets  obligations  and 
promises  to  clients  (customers)  is  indicative  of  commitment 
to  quality  (Armentrout,  1986:144). 

Process  Participation 

Orientation  Orientation 

\  Z1 2 3 

Performance 

Management 


V 

Customer 

1)  Internal 

2)  External 

Figure  2.  The  Viewpoint  of  Performance  Management 


Secondly,  performance  management  stresses  a  process 
orientation.  The  processes  which  comprise  the  business  of 
the  organization  are  the  emphasis  for  improvement  This 
requires  identifying  these  processes,  and  determining  who  is 
responsible  for  their  successful  completion  (Kacker, 
1988:41).  Opportunities  for  improving  the  way  these 
processes  are  done  are  identified,  no  matter  how  small.  It 
is  the  combination  of  these  small  improvements  which  will 
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eventually  produce  significant  results  (Department, 

1989a : 10- 1 1 ) .  For  example,  the  engineering  branch  may 
identify  the  review  of  design  drawings  for  new  buildings  as 
one  process  for  which  the  Chief  of  Design  is  the  person  in 
charge.  This  design  review  process  would  then  be  examined 
for  potential  improvements. 

Finally,  performance  management  needs  to  be 
participation  oriented.  The  basis  for  this  is  that  no 
manager  has  all  the  best  ideas.  The  input  of  the  workers 
involved,  those  closest  to  the  action,  must  be  included  to 
truly  achieve  the  performance  improvements  desired.  When 
employees  become  aware  that  their  ideas  are  being  listened 
to,  they  will  want  to  contribute.  Listening  to  what  they 
have  to  say  will  be  beneficial  (Department,  1989a: 14). 

Using  such  a  participative  approach  with  engineers  is 
especially  productive,  due  to  the  training  and  talents  that 
these  individuals  typically  have  (Stewart  and  Calloway, 
1982:113).  Engineers  are  familiar  with  the  problem  solving 
process  used  in  their  design  work.  The  management  skills 
engineers  gain  in  leading  design  teams  is  also  an  advantage. 

Specific  Problem 

Previous  research  has  examined  the  initial 
implementation  of  TQM  in  the  Operations  and  Maintenance 
Branch  of  a  Civil  Engineering  Squadron  (Wertz,  1989).  This 
research  will  pursue  a  subject  suggested  by  this  previous 
research:  Development  of  a  training  course  and  handbook 
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detailing  a  method  to  implement  a  performance  management 
framework  in  the  Contract  Engineering  Branch.  The  design 
and  steps  to  implement  such  a  framework  in  the  Engineering 
Branch  at  Wr ight-Patterson  AFB ,  Ohio  will  be  documented. 

The  change  in  attitude  toward  quality  accompanying  this 
implementation  will  be  measured  using  a  quality  survey. 

This  will  help  evaluate  the  degree  to  which  a  transformation 
of  viewpoint  is  needed  before  performance  management  can  be 
ef  f ecti ve . 

The  Contract  Planning  and  Engineering  Branch  of  an  Air 
Force  Civil  Engineering  Squadron  (Engineering  Branch)  is 
responsible  for  planning,  designing,  and  supervising  the 
contractual  performance  of  projects  which  maintain  or 
improve  the  physical  plant  of  an  Air  Force  base.  This 


Duties  : 

1)  Cost  Est.  1)  Design  Dwgs .  1)  Inspection 

2)  Fiscal  Yr .  2)  Specifications  2)  Change  Orders 

3)  Requiremts .  3)  Detailed  Cost 

Estimate 

Figure  3.  The  Contract  Project  Process 

physical  plant  includes  the  roads,  utilities,  facilities, 
and  grounds.  A  significant  difference  between  the 
Engineering  Branch  and  the  Operations  and  Maintenance  Branch 
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previously  studied  is  that  the  Engineering  Branch  employs 
more  professional  and  technical  personnel,  whereas  the 
Operations  Branch  employs  more  craftsmen  and  administrative 
people.  While  both  have  a  service  orientation,  this 
difference  in  personnel  may  lead  to  differences  in  use  of 
the  performance  management  concepts.  This  research  will  not 
repeat  the  efforts  of  the  previous  research.  Rather,  a 
framework  for  the  implementation  of  performance  management 
specifically  tailored  to  the  Engineering  Branch  of  Air  Force 
Civil  Engineering  will  be  developed. 

fixation  Purpose 

The  Department  of  Defense  (DoD)  has  been  instructed  to 
implement  TQM  across  all  organizations  in  the  armed  forces 
(Department,  1989b:2).  This  approach  to  management  has  been 
found  to  produce  significant  improvement  of  Job  satisfaction 
among  workers,  productivity  levels,  cost  reductions,  and 
customer  satisfaction.  The  Japanese  were  the  first  to  adopt 
TQM  on  a  large  scale  and  to  demonstrate  its  potential 
(Walton,  1986:25-32,122-123).  The  United  States  Government, 
recognizing  the  benefits  to  be  gained  for  its  own  military, 
decided  to  have  federal  organizations  initiate  such 
management  programs.  Air  Force  Civil  Engineering  is  thus 
responsible  to  accomplish  this  transformation  within  its  own 
ranks.  The  TQM  concepts  must  be  adapted  into  a  framework 
for  performance  management  tailored  to  the  civil  engineering 
environment . 
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This  research  will  closely  examine  the  actual 


implementation  of  a  performance  management  framework  in  an 
Air  Force  Civil  Engineering  environment.  It  will  document 
the  steps  taken,  the  difficulties  encountered,  and  the 
results  obtained.  A  training  course  and  handbook  will  be 
developed  specifically  designed  for  the  Engineering  Branch 
to  aid  in  the  implementation  of  performance  management. 

This  research  will  further  measure  the  extent,  if  any,  to 
which  the  attitudes  of  the  personnel  associated  with  this 
implementation  in  the  squadron  are  altered  toward  their  role 
in  performing  well.  The  extent  of  any  improvements  in  the 
processes  underlying  the  work  of  the  squadron  will  also  be 
documented . 


Table  1.  Research  Goals 


1)  Document  and  analyze  the 
implementation  of  performance 
management . 

2)  Develop  a  training  course  and 
handbook  to  aid  in  efforts  to  implement 
performance  management  in  other 

engi  neering  branches. 

3)  Measure  any  change  in  attitude  toward 
quality  accompanying  implementation  of  a 
performance  management  framework. 


This  information  3hould  prove  valuable  to  Air  Force 
Civil  Engineering  managers  preparing  to  implement 
performance  management  in  their  own  organizations.  In 


addition,  a  statement  can  be  made  of  how  effective  the 
implementation  is  in  the  target  Civil  Engineering  Squadron. 
To  the  extent  allowed  by  the  research  method  adopted,  this 
information  may  be  extended  to  the  potential  effectiveness 
of  performance  management  in  other  squadrons.  This 
information  will  be  the  foundation  for  a  training  course  and 
a  handbook,  recommending  methods  and  options  for 
implementing  a  performance  management  framework.  This 
training  course  and  handbook  will  facilitate  the  use  of  the 
lessons  learned  in  this  research. 

Scope  and  Limitations 

The  target  for  research  is  the  Engineering  and  Contract 
Planning  Branch  of  an  Air  Force  Civil  Engineering  Squadron. 


Air  Force 

C JLvi  1^  Engineering  Squadron 


1 

1 

1 

1 

• 

l 

1 

i 

l 

Housing 

Industrial  Eng. 

Other  Branches 

—  — - 

Operations  &  Engineering 

Maintenance 

Fire 

Prevention 

S -  “ 


Previous  This 

Research  Research 


Figure  4.  Civil  Engineering  Organizational  Chart 
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Research  regarding  implementation  of  TQM  in  the  Operations 
and  Maintenance  Branch  was  previously  performed  (Wertz, 
1989).  Housing,  Industrial  Engineering,  and  other  branches 
will  not  be  addressed.  The  Engineering  and  Contract 
Planning  Branch  is  the  largest  of  the  branches  not  included 
in  the  previous  research,  and  has  therefore  been  chosen  as 
the  next  branch  to  be  concentrated  on.  In  addition,  the 
researcher  is  familiar  with  the  working  of  this  branch, 
facilitating  this  research. 


I2¥®§li£§kive  Questions 

In  order  to  accomplish  the  purpose  for  this  research, 
the  following  specific  questions  will  need  to  be  answered: 

1.  How  has  TQM  been  utilized  in  other  similar 

organizations?  What  benefits  have  been 
achieved?  What  changes  need  to  be  made  to 
adapt  TQM  to  the  Contract  Engineering  Branch? 

2.  What  opportunities  and  needs  exist  within  the 

Engineering  Branch  for  improvement? 

3.  What  training  of  personnel  is  used  prior  to  or 

during  implementation  of  TQM? 

4.  What  obstacles  are  encountered  during  the 

initial  implementation  of  TQM? 

5.  What  short  range  benefits  can  be  measured  after 

implementation  of  TQM? 

6.  What  outside  resources  (i.e.  private 

contractors  or  procured  training)  are  needed 
to  facilitate  the  performance  management 
ef  fort? 

7.  How  do  attitudes  toward  quality  of  work  change 

among  employees  involved  in  performance 
management? 

8.  How  is  a  performance  management  framework  used 

to  guide  improvement  efforts  in  the 
particular  situation  of  an  Engineering 
Branch? 

9.  What  guidance  do  Engineering  Branch  managers 

need  to  design  and  implement  a  performance 
management  framework  in  their  organizations? 


1  1 


Organization  of  Presentation 

Chapter  I  presents  the  introduction  to  the  subject  for 
research  and  a  description  of  the  problem  and  investigative 
questions.  The  purpose  and  scope  are  given. 

Chapter  II  presents  a  review  of  the  literature 
pertinent  to  the  subject  of  research.  The  chapter  is 
divided  into  topics  relevant  to  performance  management 
and  its  implementation  in  an  engineering  environment. 

Chapter  III  describes  the  methodology  used  to  answer 
the  investigative  questions.  Reasons  for  selecting  the 
particular  methodology  are  also  given. 

Chapter  IV  summarizes  the  data  obtained  through  the 
research.  Tables  are  included  to  consolidate  the  results. 

Chapter  V  explains  the  results  and  draws  appropriate 
conclusions  from  the  data,  and  makes  recommendations  for 
further  research. 

Summary 

Performance  management  is  a  framework  of  attitudes  and 
tools  for  managers  to  use  to  increase  the  performance  of 
their  organizations.  It  stresses  customer  satisfaction, 
process  improvement,  and  employee  participation.  Air  Force 
Civil  Engineering  Squadrons  have  been  instructed  to 
implement  TQM  to  make  better  use  of  the  scarce  resources 
available  and  to  better  accomplish  their  mission  of  facility 
support.  Thi3  research  will  examine  and  document  the 
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implementation  of  the  TQM  concepts  in  a  Civil  Engineering 
Squadron  environment,  record  the  associated  results,  and 
propose  a  training  course  and  handbook  to  assist  managers  in 
implementing  a  framework  for  performance  management  in 
their  squadrons. 
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II.  Review  of  Literature 


Overview 

This  chapter  will  discuss  current  published  information 
on  the  subject  of  implementing  a  system  for  performance 
analysis,  improvement  and  management.  Much  of  this 
literature  focuses  on  Total  Quality  Management  (TQM)  and  its 
use  in  organizations  similar  to  civil  engineering.  TQM  is  a 
management  philosophy  which  stresses  continuous  improvement 
of  the  processes  which  make  up  any  organization’s  work. 

Several  issues  must  be  resolved  in  order  for 
performance  management  to  be  successfully  implemented  in  an 
organization.  Information  will  be  presented  pertaining  to 
these  issues,  which  are 

1.  Engineering  Environment  Distincti ves/Di f f iculties 

2.  Target  of  Improvement 

3 .  Measurement 

4.  Degree  of  Employee  Involvement 

5.  Extent  and  Nature  of  Training 

6.  Management  Commitment 

7.  Process  Improvement 

8.  Implementation 

Environment  inct iveS/Di f f icul ties 

Improvement  in  the  engineering  design  of  construction 
projects  has  not  received  the  attention  that  improvement  in 
manufacturing  production  techniques  has.  Total  Quality 
Management  principles  have  the  potential  for  astounding 
increases  in  productivity,  quality,  and  labor  motivation  in 
the  manufacturing  environment.  One  estimate  suggests  that 
no  less  than  65%  of  a  service  industry’s  production  is  lost 
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to  extra  steps  in  the  work  processes  needed  to  correct 
errors  (Kacker,  1988:40).  Yet  the  potential  exists  to 
use  TQM  for  greatly  increasing  productivity  and  decreasing 
cost  through  better  designs. 


The  literature  reveals  many  obstacles  preventing  wider 
use  of  TQM.  Construction  and  design  are  oriented  toward 
individual  projects,  inhibiting  improvements  from  being 
shared  from  one  project  to  another.  The  frequency  with 
which  owners  or  users  change  their  minds  about  what  they 
want  makes  it  difficult  to  accurately  evaluate  the 
effectiveness  of  the  design  itself  (Gilly  and  others, 
1987:428,430-1).  Another  impediment  is  that  engineering 
design  initially  provides  information.  When  the  output  of 
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the  work  is  a  physical  object  which  can  be  counted  and 
inspected,  evaluation  of  the  output  is  simplified.  It  is 
much  more  difficult  to  objectively  evaluate  a  conceptual 
design:  against  what  standard  should  it  be  measured 
( McGeorge ,  1988:360)? 

Target  of  Improvement 

Focus  o_f  Improvement.  By  its  very  name,  one  would 
surmise  that  the  focus  for  improvement  of  total  quality 
management  is  quality.  But  what  is  quality?  And  is  that 
indeed  the  focus7  The  Department  of  Defense,  when  mandating 
TQM,  defined  quality  as  the  degree  to  which  the  customer  is 
satisfied  (Department,  1989a:7).  Quality  in  the  civil 
engineering  environment  is  further  defined  as  meeting  the 
declared  requirements  of  the  owner,  designer,  constructor, 
and  regulating  bodies  (ASCE,  1988:1).  Above  all  others, 

Improved  +  Customer  =  Quality 

Performance  Satisfaction  Management 

Figure  6.  The  Performance  Management  Equation 

quality  is  the  greatest  single  factor  which  affects  the 
profitability  of  a  concern  (Townsend  and  Gebhardt ,  1989:18). 

While  improving  the  quantity  of  production  is  important,  the 
quality  produced  in  engineering  work  is  most  important 
(Takei ,  1981b:24) . 
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Productivity,  being  the  ratio  of  output  produced  to  the 
corresponding  input  used,  is  proposed  as  an  alternate  focus 
of  improvement  (Stewart  and  Calloway,  1982:110,  Sumanth  and 
Yavuz ,  1983:260).  In  his  work  to  improve  engineering 

productivity  in  the  Tennessee  Valley  Authority,  Daryl 
Armentrout  suggests  the  broader  index  of  performance  is 
another  possible  focus  for  improvement.  Performance 
includes  efficiency,  effectiveness,  profit,  innovation,  and 
working  conditions  in  addition  to  quality  and  productivity. 
These  seven  components  of  performance  are  equally  applicable 
to  not  for  profit  public  agencies.  Profitability  must  be 
changed  to  budgetabi 1 i ty ,  defined  as  the  degree  to  which  the 
organization  accomplishes  its  goals  and  objectives  and  keeps 
its  budget  (Sink  and  Tuttle,  1989:185).  The  measure  of 
effectiveness  is  given  as  how  well  commitments  to  the 
customer  are  satisfied  (Armentrout,  1986:142,144).  This 
then  comes  back  to  customer  satisfaction  as  suggested  by 
DoD. 


Table  2.  Components  of  Performance 


Ef  f ectiveness 
Qual i ty 
Efficiency 
Productivity 
Quality  of  Work  Life 
Innovation 
Budgetabi 1 i ty  or 
Prof i tabi 1 i ty 
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Lieutenant  General  John  M.  Loh ,  as  Commander  of  the  Air 


Force  Aeronautical  Systems  Division,  lists  'delighting  the 
customer'  as  one  of  the  guiding  principles  of  that 
organization.  He  includes  the  contractors  retained  by  the 
Air  Force  among  those  customers.  The  engineering  designs 
and  documents  prepared  by  the  government  are  the  product 
that  needs  to  be  improved  to  satisfy  that  customer  (Loh, 
1989).  Another  customer  for  the  engineering  organization  is 
the  field  worker  (Takei ,  1981b:23).  It  is  imperative  that 

the  needs  of  these  customers,  as  well  as  those  of  the 
ultimate  users  of  the  facilities,  be  identified  and  goals 
set  to  satisfy  these  needs  (Kacker,  1988:40). 


Field 


tf 


Customers 
of  Engineering 


Workers 


I 

Users 


Contractors 


Figure  7.  Civil  Engineering  Customers 


Method  of  Improvement .  The  obstacles  to  performance 
management  implementation  in  the  engineering  environment 
discussed  above  need  to  be  overcome  for  the  effort  to  be 
effective.  The  engineers  themselves  are  the  most  effective 
resource  in  overcoming  these  obstacles.  One  suggestion  is 
to  establish  a  team  of  engineers  to  develop  quality  and 
productivity  improvements.  The  team  determines  the  critical 
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factors  affecting  the  work,  and  establishes  a  theme  or  goal 
for  improvement.  A  measurement  formula  must  then  be  devised 
to  provide  data  for  planning  the  improvement  method.  Once 
appropriate  and  meaningful  measures  are  established,  methods 
are  attempted  to  improve  the  system  until  success  is 
achieved.  Then  a  new  theme  for  improvement  is  decided  upon 
and  the  steps  repeated  (Takei ,  1986:93). 

Measurement 

Meaningful  Measures .  Measuring  the 
productivity  or  quality  of  an  engineering  concern  is 
difficult.  The  output  of  engineering  is  often  one  of  ideas, 
a  product  not  easily  quantified.  An  ideal  product  measure 
would  be  to  relate  total  output  to  all  associated  input 
(Sumanth  and  Yavuz ,  1983:261) .  This  total  factor  ratio 

should  be  applied  where  costs  permit.  However,  attaining 
this  kind  of  total  measure  may  not  only  be  expensive,  it  may 
be  impossible  due  to  the  inability  to  assign  the  engineer’s 
input  (time)  to  a  particular  output.  Two  other  levels  of 
measurement  are  partial  factor  and  single  factor  ratios.  A 
single  factor  ratio  is  developed  by  selecting  some  lone 
factor  believed  to  accurately  represent  the  total  input  or 
output.  This  single  factor  is  then  used  in  place  of  the 
overall  total  in  computing  productivity.  Partial  factor 
models  use  more  than  one  factor,  but  fall  short  of  being 


total  factor  evaluations. 


Surrogate  measures,  which  link  productivity  to  more 
easily  observed  characteristics,  such  as  utilization  of 
equipment  capacity,  have  been  proposed  as  an  alternative  to 
the  preceding  actual  product  measures  (Stewart,  1978:34, 
Armentrout,  1936:142).  Process  measures  are  another 
alternative,  where  the  method  used  to  achieve  the  end 
product  is  evaluated.  The  measurement  formula  devised  must 
correspond  to  the  effectiveness  of  the  actual  product,  be  it 
design  or  facility,  not  just  activity  related  to  it 
(McGeorge,  1988:352).  The  points  in  the  processes  where 
evaluations  are  to  be  made  is  likewise  important. 

Convenient  points  of  assessment  are  the  interfaces  between 
separate  subprocesses  which  make  up  the  whole  (Kacker, 
1988:41).  In  some  cases,  surrogate  or  process  measures  are 
much  more  easily  attained  than  the  product  measures,  due  to 
the  previously  mentioned  difficulties  in  quantifying  product 
attributes . 

Absolute  measurement  systems  evaluate  performance 
compared  to  some  externally  defined  standard.  The 
Engineered  Performance  Standards  (EPS)  used  in  evaluating 
craftsmen’s  work  are  of  this  type.  Such  absolute  scales  are 
difficult  to  develop  for  engineering  design.  Another 
measure  of  interest  would  be  comparing  how  well  the  promises 
and  commitments  to  clients  were  kept  (Armentrout,  1986:143). 
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Table  3. 


Measurement 


Levels  of  Measures 


Total  Factor 
Partial  Factor 
Single  Factor 

T^Ees  of  Measures 

Product 

Process 

Surrogate 

Classes  of  Measures 

Relative 

Absolute 

Comparative  measures  have  also  been  proposed  as  a 
solution  to  the  problem  of  establishing  absolute  scales  for 
evaluating  the  productivity  and  quality  of  engineering 
organizations.  Evaluation  might  be  made  on  the  basis  of 
volume  of  output  supported  against  the  most  competitive 
engineering  firms  (Takei ,  1981a: 13)  and  against  government 
engineering  departments.  Alternatively,  performance  trends 
of  the  subject  organization  might  be  tracked  over  time  to 
assess  progress  and  improvement  (Stewart  and  Calloway, 

1982 : 1  15)  . 

Ro  1. e  of  Measurement.  Measurement  is  a  fundamental  tool 
to  be  used  in  the  performance  management  framework.  A 
measurement  and  feedback  system  must  be  created  which  allows 
the  processes  of  interest  to  be  monitored  and  improved 
(Dingus  and  Hrivnak,  1988:30).  A  system  to  measure 
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performance  is  essential  in  order  to  understand  the  causes 
of  variation  in  work  processes  which  lead  to  job  breakdowns 
(Hacquebord  and  Scholtes,  1988a:29).  Measurement  of 
productivity  is  a  necessary  ingredient  in  any  effort  to 
improve  productivity  (White  and  Austin,  1989:371).  Accurate 
measurement  of  performance  is  useful  for  other  purposes  as 
well.  Once  improvement  actions  have  been  implemented, 
measuring  the  impact  of  those  actions  can  permit  errors  in 
the  improvement  actions  to  be  fixed  before  proceeding 
(Edosomwan,  1987:67).  These  modi f icat i ons  generated  from 
the  feedback  from  improvement  efforts  can  result  in  even 
greater  performance  improvements.  The  measurement  of 
improved  performance  can  be  a  record  to  show  to  management 
to  convince  them  of  the  value  of  the  performance  management 
effort  (Gilly  and  others,  1987:436).  If  performance 
management  is  competing  with  other  opportunities  for  scarce 
resources,  this  evidence  of  success  can  be  important  in 
ensuring  the  continuance  of  the  effort. 

Table  4.  Roles  of  Measurement 

1)  Evaluate  Processes 

2)  Improvement  Feedback 

3)  Sell  Management 

Degree  of  Employee  Involvement 

Performance  management  stresses  the  employees’ 
involvement  in  management.  Individuals  from  all  levels  of 
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the  organization,  including  the  line  workers,  3hare  in 
decision  making.  This  employee  involvement  can  be  used  in 
first  identifying  the  processes  to  be  improved  (Hacquebord 
and  Scholtes,  1988b:46).  The  employees  affected  can  then  be 
included  in  brainstorming  possible  solutions  to  problems  and 
strategies  for  improvement  (Department,  1989a: 11).  Group 
interaction  can  also  be  used  effectively  in  establishing  the 
measures  to  be  used  in  evaluating  quality  and  productivity 
(Stewart  and  Calloway,  1982:114). 

Table  5.  Employee  Involvement  Possibilities 

1)  Identify  processes  for  improvement 

2)  Brainstorm  problems  and  solutions 

3)  Establish  measures 

Two  formats  for  employee  participation  are  the  work 
group  and  the  quality  circle.  Quality  circles  are  comprised 
of  workers  from  a  single  work  setting  within  the 
organization,  and  often  include  their  immediate  supervisors 
(Steel  and  Lloyd,  1988:3).  Work  groups  are  temporary  teams 
assembled  from  members  of  the  several  work  settings  which 
comprise  a  process  within  the  organization.  A  particular 
team  will  continue  to  meet  until  the  improvement  desired  to 
a  process  has  been  accomplished,  and  will  then  be  dissolved 
(Department,  1989a: 15). 
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Functional  Area  Quality  Circles 


Mul ti - 

Function 

Process 

Action 

Team 


Figure  8.  Two  Types  of  Teams 

Extent  and  Nature  of  Training 

Extent.  Training  of  the  employees  and  managers 
involved  in  TQM  is  important.  One  approach  to  assure 
training  does  not  miss  anyone  is  to  train  everyone  at  the 
outset  (HQ  AFLC ,  1989a: 10).  A  major  risk  of  doing  this  is 
that  by  the  time  the  last  people  are  being  trained  the  first 
ones  will  be  losing  their  enthusiasm  for  the  program  (Roth, 
1989:28).  A  better  approach  is  to  get  started  into  the 
quality  transformation  and  then  provide  the  training 
required  at  the  time  needed  (Hacquebord  and  Scholtes, 

1988b: 48) . 

Nature.  A  cultural  transformation  needs  to  take  place 
in  an  organization  for  a  performance  management  framework  to 
be  effective.  Training  is  an  important  vehicle  for 
accomplishing  this  change  in  culture.  The  new  attitude 
needed  is  one  where  every  member  of  the  workplace  thinks 
improvement,  and  brings  a  pride  and  enthusiastic  approach  to 
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their  improvement  efforts  (Ryan  and  Sebastanel 1 i ,  1987: 
331-2).  A  problem  with  many  training  programs  is  that  not 
enough  attention  is  paid  to  this  cultural  change;  too  much 
emphasis  is  placed  on  teaching  the  tools  of  performance 
management  (Warmington,  1988:36).  The  team  leader  and 
facilitator  should  be  the  only  ones  to  receive  training  in 
the  use  of  these  technical  tools  at  the  outset;  the  team 
members  can  pick  up  what  they  need  in  the  actual  on-the-job 
problem  solving  (Gilly  and  others,  1987:429).  The  tools  of 
particular  use  in  performance  management  are  measurement, 
control,  planning  and  cause/effect  improvement  techniques 
(Edosomwan,  1987:67). 


Cultural  Change  Orientation 
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Speci f ic 
Tools 
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Figure  9.  Two  Different  Training  Objectives 


??§ 2 §6 £!?£!!£  Commitment 

The  success  of  the  performance  management  effort  lives 
or  dies  on  the  support  and  direction  provided  by  management. 
The  improvement  program  must  begin  with  management 
commitment  and  end  with  management  rewarding  the 
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participants  (Edosomwan,  1987:67-8).  Management  cannot 
delegate  responsibility  for  performance  management  to  a 
staff  group  or  quality  coordinator  and  then  forget  about  it 
this  communicates  that  it  was  not  important  to  begin  with 
(Warmington,  1988:26).  Continued  management  commitment  is 


One  of  management’s  important  roles  is  to  direct  the 
improvement  effort.  The  other  is  to  support  it. 

Figure  10.  Roles  of  Management 

needed.  When  the  initial  momentum  has  worn  off,  only 
management  can  jump  in  to  keep  the  hope  alive  (HQ  AFLC , 
1989b:7) .  The  challenge  to  management  is  to  spur  the 
organization  on  to  improvement,  and  to  provide  creative 
ways  to  get  there  (Dingus  and  Hrivnak,  1988:29). 
Management’s  input  is  critical  in  deciding  which  aspects  of 
the  workplace  are  most  important  in  directing  the 
improvement  program  (Stewart,  1978:37).  The  potential  for 
the  greatest  improvements  is  at  the  early  conception  and 
development  phases  of  the  work  process;  these  are 
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management’s  domain,  and  need  their  attention  to  achieve  all 
the  improvements  possible  (McGeorge,  1988:358). 


Figure  11.  Process  Improvement  Potential 

When  the  performance  improvement  team  has  developed  its 
proposed  solutions,  attention  must  be  returned  to 
management.  A  management  presentation  is  an  ideal  way  to 
make  this  transition.  The  team  prepares  their  solutions  in 
the  form  cf  a  fifteen  to  twenty  minute  presentation.  Top 
management  and  the  department  heads  affected  are  invited. 

The  presentation  should  include  a  specific  request  for 
action  by  management.  This  high  visibility  presentation 
serves  not  only  to  inform  management  of  the  results  of  the 
team’s  effort,  but  it  also  allows  the  team  members  to 
conclude  their  activities  on  an  exciting  note.  (Todd,  1990) 


27 


Pro ce 3 s  Improvement 

Identification  of  Process  for  Improvement .  Before  the 
processes  which  make  up  the  way  an  organization  does 
business  can  be  improved,  those  processes  need  to  be  made 
visible  and  mapped  out  (Kacker,  1988:41).  Asking  workers 
whom  within  the  organization  they  depend  on  can  provide 
clues  to  internal  provider  relationships.  Discovering  whom 
they  provide  service  to  can  help  to  uncover  the  customers  in 
these  processes.  Determining  these  providers  and  customers 
helps  establish  the  interfaces  between  subprocesses 
(Hacquebord  and  Scholtes,  1388a: 29).  In  an  engineering 
environment  the  overall  process  is  made  up  of  three  distinct 
phases:  planning,  analyzing,  and  coordination  (Stewart  and 

Calloway,  1982,110). 


Figure  12.  Phases  of  the  Engineering  Process 

Scope  of  Process.  Improvement  of  engineering  design 
will  be  most  effective  when  the  improvement  efforts  are 
aimed  at  the  entire  design  and  construction  process,  from 
conception  to  delivery  of  the  completed  project  to  the 
customer.  Quality,  an  important  component  in  design,  is 
dependent  on  the  quality  with  which  each  individual  step  in 
the  design  process  is  completed,  from  initial  surveys  and 
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information  gathering  through  design  to  production  and 
testing  (Takei ,  1986:92).  Much  attention  has  been  given  to 
improving  the  technical  design  and  drafting  steps  in  the 
process,  through  innovations  such  as  standardization  of 
details  and  CAD.  However,  vast  improvements  at  minimal 
costs  can  best  be  made  in  the  early  concept  stages  of 
project  design  (McGeorge,  1988:353,357).  Having  started 
with  design,  improvement  efforts  must  continue,  through 
construction,  until  the  facility  is  delivered  to  the  user  to 
be  effective  (Gilly  and  others,  1987:428). 

Strategy  for  Improvement.  Once  the  process  is 
identified  a  strategy  for  improvement  needs  to  be  developed. 


Figure  13.  The  Deming  Wheel 

An  effective  model  to  use  for  this  strategy  is  the  Deming 
Wheel,  which  consists  of  planning  ways  to  improve,  doing 
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what  is  needed  to  get  started,  checking  the  effectiveness  of 
the  activity,  then  acting  to  complete  the  planned 
improvement,  and  starting  all  over  again  with  planning  a  new 
improvement  (Hacquebord  and  Scholtes,  1988b:46).  An 
alternate  to  the  Deming  Wheel  is  the  productivity  cycle, 
defined  as  measuring  current  producti vity ,  evaluating  the 
results,  planning  how  to  improve  the  level  of  productivity, 
then  implementing  the  improvements,  and  beginning  again  with 
measuring  productivity  (Sumanth  and  Yavuz ,  1983:261). 

2?®  Station 

E^EiAi^ators .  The  facilitator  is  an  important  part  of 
the  performance  management  team.  The  job  of  the  facilitator 
is  to  observe  the  participation  of  the  members  of  the  team, 
and  to  direct  them  in  the  problem  solving  process.  Toward 
this  end,  the  person  selected  to  facilitate  the  team  should 
attend  a  three  or  four  day  training  course  to  learn  what  to 
look  for  and  the  tools  to  suggest  (Gilly  and  others, 
1987:429).  The  facilitator  can  have  a  profound  impact  on 
the  team’s  effectiveness;  "...an  effective  facilitator  can 
unlock  the  energy  and  intelligence  of  the  group  to  solve 
problems'  (Fishman,  1989:22).  The  person  facilitating  the 
team  may  not  be  the  boss  of  anyone  on  the  team,  or 
participation  will  suffer  (Roth,  1989:30). 

The  duties  of  the  facilitator  start  with  planning  each 
meeting.  Drafting  and  posting  an  agenda  to  outline  the 
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goals  and  schedule  of  the  meeting  can  help  keep  the  group 
time  focused.  This  motivates  the  team  members  to  apply 
themselves  to  the  subject  and  goals,  by  being  aware  of  what 
needs  to  be  accomplished.  Further  duties  of  the  facilitator 
are  to  get  the  other  group  members  involved  in  the  work. 
Tasks  need  to  be  delegated  to  others  as  much  as  possible. 
These  tasks  might  include  keeping  minutes,  recording 
important  thoughts  on  an  easel  pad  or  overhead  for  the  group 
to  refer  to,  or  developing  charts  and  graphs  to  illustrate 
information.  (Todd,  1990) 


Figure  14.  The  Facilitator  is  the  Key 


Me •  When  TQM  is  first  implemented  in  an 
organization,  an  awkward  start  is  to  be  expected. 

Management  needs  to  get  the  ball  rolling  and  establish  some 
momentum  in  order  to  get  through  this  start-up  (Hacquebord 
and  Scholtes,  1988a : 28 , 3 1 -32 ) .  One  suggestion  is  to  try  a 
high  visibility  and  easily  understood  process  for  the  first 
improvements  (Townsend  and  Gebhardt,  1989:20).  Another 
possibility  is  distributing  articles  trumpeting  the 
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successes  of  performance  management  prior  to  kicking-off  the 
effort  itself  (Ryan  and  Sebastanel 1 i ,  1987:328). 

Planning  for  the  improvement  effort  is  essential  before 
diving  in.  One  of  the  main  reasons  performance  management 
fails  is  due  to  a  lack  of  strategic  planning  for  the  change 
(Vogt  and  Hunt,  1988:96).  Caution  is  in  order  to  make  sure 
a  framework  is  well  thought  out  before  starting  any  kind  of 
team  improvement  efforts  (Roth,  1989:29).  Use  of  a  steering 
group  is  recommended  to  list  the  key  processes  in  the 
workplace  and  assign  facilitators  and  improvement  teams  to 
tackle  them  (Department,  1989a:35).  Part  of  this  initial 
planning  is  to  identify  who  the  informal  leaders  in  the 
organization  are,  and  develop  a  strategy  to  use  their  help 
in  implementing  the  performance  management  framework 
(Hacquebord  and  Scholtes,  1988a:31). 

Small  problems  are  a  good  place  to  start  on  a  brand  new 
performance  management  framework  (Gilly  and  others, 

1987:435).  This  way  some  early  small  successes  can  help 
bolster  enthusiasm  for  later  larger  projects.  Follow  a 
process  of  identifying  a  theme  for  improvement,  determining 
the  important  factors  affecting  that  theme,  and  then  seeking 
to  measure  and  improve  those  factors.  The  theme  may  be 
reducing  customer  complaints,  shortening  turn-around  time, 
or  similar  goals  (Takei ,  1986:93).  The  factors  must  be  well 

understood,  and  the  problems  correctly  diagnosed,  before 
applying  solutions,  or  lasting  improvements  may  be  missed 
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(Warmington,  1988:40).  Management  must  act  on  the 
recommended  actions  for  implementation  from  the  team,  or  the 
team  will  give  up  due  to  the  perceived  futility  of  their 
work  (Vogt  and  Hunt,  1988:96). 


Identify 

Improvement 

Theme 


Determine 

Important 

Factors 


Measure  and 
Improve 
Factors 


^=3  <^=;  13  Repeat  -J 


Figure  15.  Improvement  Theme  Methodology 


Summary 

TQM  is  a  management  philosophy  which  has  been  used  by 
both  manuf aotur ing  and  service  organizations  to  improve  the 
way  they  do  business.  The  United  States  Department  of 
Defense  has  mandated  that  all  military  organizations  adopt 
TQM  techniques  in  order  to  cut  costs,  improve  productivity 
and  job  satisfaction,  and  provide  better  customer 
satisfaction.  Air  Force  Engineering  Branches  are  among 
those  affected.  Because  engineering  generally  deals  with 
ideas  instead  of  tangible  products,  the  way  a  performance 
management  framework  will  be  implemented  and  its  success 
measured  needs  to  be  carefully  considered. 

The  literature  reviewed  in  this  paper  summarized  the 
current  thought  on  how  to  accomplish  this.  Managers  need  to 
determine  whether  to  stress  quality,  productivity,  customer 
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satisfaction,  or  some  aggregate  of  these.  The  processes  for 
which  they  are  responsible  must  be  identified  and  strategies 
prepared  to  improve  them.  It  is  important  to  establish 
meaningful  measures  for  productivity,  directly  or  through  a 
surrogate  measure.  Managers  must  determine  where  to  make 
these  measurements  and  how  to  judge  the  results  obtained.  A 
plan  for  the  degree  and  nature  of  involvement  for  employees 
needs  to  be  decided  upon,  as  well  as  how  and  when  to  train 
those  employees.  Managers  must  also  be  sensitive  to 
creating  the  initial  enthusiasm  required  to  get  their 
programs  off  the  ground. 
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III.  Research  Method 


Overview 

A  case  analysis  of  the  2750  Civil  Engineering  Squadron 
Engineering  Branch  located  at  Wr ight-Patterson  Air  Force 
Base,  Ohio,  was  the  primary  method  used  to  address  the 
specific  problem  and  investigative  questions  previously 
described.  One  reason  this  Engineering  Branch  was  chosen 
was  because  of  its  proximity  to  the  researcher,  providing 
for  ease  of  making  the  required  observations  for  a  case 
study.  A  second  reason  is  because  its  Major  Command,  Air 
Force  Logistics  Command  (AFLC) ,  has  been  emphasizing  TQM, 
so  that  the  Branch  wa3  already  interested  in  performance 
management,  though  no  steps  had  been  taken  to  implement  any 
framework  for  improvement  prior  to  this  research. 

Case  Literature  Quality 

Analysis  Review  Survey 

^  I  ^ 


Synthesis 


Training  Course  Conclusions,  Hypotheses 

and  Handbook  and  Recommendations 

Figure  16.  Research  Method 

In  addition,  a  survey  instrument  called  the  Quality 
Questionnaire  was  used  to  determine  any  shift  in  attitude 
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toward  quality  that  accompanied  the  implementation  of 
performance  management.  Statistical  tests  were  utilized  to 
determine  if  any  significant  change  occurred. 

The  insights  gathered  from  the  careful  observations  of 
the  case  analysis  were  combined  with  knowledge  obtained  from 
a  thorough  review  of  the  literature  to  develop  a  training 
course  and  handbook  for  engineering  managers  interested  in 
implementing  a  framework  for  performance  management. 
Hypotheses  and  recommendations  regarding  the  implementation 
of  performance  management  in  Engineering  and  Contract 
Planning  Branches  of  Air  Force  Civil  Engineering  Squadrons 
were  generated. 

Literature 

A  comprehensi ve  review  ot  the  literature  published  on  a 
subject  is  a  crucial  element  to  any  research.  It  is  in  the 
literature  that  researchers  find  the  current  state  of 
knowledge  on  a  topic  and  thereby  can  assure  that  they  are 
not  going  over  the  same  ground  which  has  already  been 
covered . 

Concurrent  with  the  case  analysis  described  below,  a 
representative  review  of  published  literature  on  the  subject 
of  performance  management  implementation  was  accomplished. 
Particular  emphasis  was  placed  on  recent  papers  and  articles 
in  professional  journals.  The  search  for  such  articles 
began  with  a  search  of  defense  oriented  literature  and  a 
search  of  industry  and  trade  journals.  Both  searches  were 
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assisted  by  the  Air  Force  Institute  of  Technology  library 
staff  using  on-line  computer  aids  at  the  researcher's 
request.  Beginning  from  the  references  identified  by  these 
searches,  and  continuing  with  references  cited  in  the 
bibliographies  of  these  articles,  the  literature  was 
reviewed.  Notes  of  important  information  were  taken  and 
organized  by  topic  for  use  in  comparing  to  insights  from  the 
case  analysis.  (Selltiz  and  others,  1964:53) 

Case  Analysis 

Mystification  for  Case  Analysis  Design.  The  state  of 
knowledge  regarding  performance  management  implementation  is 
still  in  its  infancy.  Very  little  has  been  published 
focusing  on  performance  management  in  American  government  or 
the  engineering  environment.  Therefore,  an  exploratory 
qualitative  research  design  is  desirable,  to  increase  this 
state  of  knowledge.  Exploratory  research  should  be  seen  as 
the  important  first  step  in  an  ongoing  research  process. 

This  process  culminates  in  the  quantitative  research 
analysis  of  hypotheses  and/or  descriptive  and  causal 
studies.  However,  proceeding  to  such  quantitative  research 
prematurely,  before  qualitative  research  has  allowed  the 
formulation  of  appropriate  hypotheses  and  research 
questions,  can  decrease  the  effectiveness  of  such 
quantitative  research.  (Selltiz  and  others,  1964:52) 
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Four  options  are  most  appropriate  for  such  exploratory 
research:  historical,  case  analysis,  delphi  technique,  and 
meta-analysis.  The  meta-analysis  was  rejected  because  of 
the  scarcity  of  published  literature  applicable  to  the 
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Figure  17.  Qualitative  Versus  Quantitative  Research 


research  problem.  The  historical  method  was  rejected 
because  most  of  the  track  record  on  performance  management 
is  from  Japan.  The  extreme  cultural  differences  between 
Japan  and  America  would  make  lessons  learned  from  the  first 
difficult  to  apply  to  the  second.  The  delphi  technique 
would  be  a  powerful  tool  to  use  in  this  situation  when  more 
experience  has  been  gained  by  people  in  America  using 
performance  management.  At  the  present  time,  however,  use 
of  an  in-depth  case  analysis  to  observe  and  report  on 
performance  management  implementation  first  hand  was 
selected . 
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The  case  analysis  is  useful  to  discover  knowledge  on  a 
subject  with  the  end  of  establishing  a  definite  hypothesis 
or  research  problem  for  further  study  (Selltiz  and  others, 
1964:50).  A  strength  of  the  case  analysis  is  its  ability  to 
provide  an  intensive  examination  of  a  single  unit.  From 
this  examination  not  only  the  broad  picture  but  the  fine 
details  can  be  recorded  and  analyzed.  Another  strength  is 
that  the  end  result  is  not  just  a  snapshot  in  time  of  the 
condition  of  the  subject.  A  record  of  the  steps  taken, 
problems  encountered,  and  corrections  made  is  preserved. 

A  principal  weakness  of  the  case  analysis  is  that  it  is 
not  easily  replicated.  The  results  obtained  are  in 
some  ways  peculiar  to  the  subject  of  the  study;  ability  to 
generalize  the  results  are  reduced.  Another  weakness  is  the 
potential  for  the  researchers/observers  to  include  their 
personal  bias  in  the  reporting.  They  must  be  ever  alert 
to  remain  objective  (Selltiz  and  others,  1964:60). 

Table  6.  Case  Analysis  Characteristics 

Strengths  of  Case  Analysis 

1)  Create  knowledge  on  subject 

2)  Provide  broad  understanding  of  subject 

3)  Preserve  record  of  case  development 

Weaknesses  of  Case  Analysis 

1)  Hard  to  replicate  results 

2)  Generalization  of  conclusions  limited 

3)  Subject  to  bias  of  researcher 
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Descr i pt i on  of  Case  Analysis.  The  Chief  of  the 
Engineering  and  Contract  Planning  Branch  was  contacted  in 
August  1989.  Arrangements  were  made  to  establish  a  team  to 
evaluate  and  improve  the  process  of  design  review.  This 
process  was  chosen  by  the  Chief  as  needing  improvement. 
Design  review  is  a  process  whereby  the  affected 
organizations  on  the  Air  Force  installation  review  the  plans 
and  specifications  for  proposed  construction  work. 

Five  individuals  were  assigned  by  the  Chief  to 
participate  on  the  team.  Each  member  was  a  representative 
of  a  particular  function  within  the  Engineering  Branch 
contributing  to  the  process.  The  individuals  chosen  were 
not  perceived  to  have  any  distinctive  aptitude  for  the  task 
over  other  members  within  the  same  function. 

The  team  met  once  a  week  for  one  hour  from  November 
1989  through  summer  1990.  The  team  followed  the  scientific 
method  for  problem  solving.  They  identified  the  steps  in 
the  process,  developed  a  list  of  problems  they  were  aware 
of,  proposed  solutions  to  these  problems,  and  formulated 
programs  to  implement  these  solutions. 

The  researcher  attended  these  meetings  as  a  facilitator 
and  observer.  The  team  was  assisted  in  following  the 
scientific  method.  A  member  of  the  Industrial  Engineering 
Branch  of  the  Civil  Engineering  Squadron,  experienced  in 
facilitating  such  a  team  endeavor,  assisted.  Important 
information  was  noted  as  it  surfaced  about  implementation  of 
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the  framework  for  performance  management.  The  ingredients 
essential  to  successful  case  analysis  research  were 
considered:  1 ) open-minded  and  careful  scrutiny;  2) accurate 
synthesis  of  the  data  (Selltiz  and  others,  1964:60).  To 
this  end,  notes  and  minutes  of  the  meeting  were  kept  and 
organized  into  topics.  These  topics  were  compared  to  those 
developed  from  the  review  of  the  literature  described  above. 
In  this  way  consistent  trends  were  established  from  both 
published  and  observed  characteristics. 

£or  •  An  important 

consideration  in  implementing  performance  management  is  the 
cultural  change,  or  adjustment  in  attitude,  which  first 
must  occur  for  tae  effort  to  be  successful  (Warmington, 
1988:36).  Observing  the  change  in  individuals’  attitude 
toward  quality  issues  such  as  leadership,  planning,  and 
customer  satisfaction  associated  with  the  case  analysis  may 
reveal  insight  into  this  cultural  change. 

A  non-equivalent  control  group  design  was  chosen  to 
augment  the  case  analysis.  Figure  3.18  shows  the  form  of 
the  quasi-experiment.  The  goal  was  to  measure  whether 
attitudes  changed  toward  quality  among  those  participating 
on  the  quality  improvement  team.  Because  the  treatment 
group  was  selected  in  a  non-random  fashion,  a  true 
experiment  was  impossible.  The  researcher  had  the  ability 
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to  test  the  treatment  group  before  and  after  the  treatment, 
and  a  control  group  at  the  same  times,  so  the  quasi- 
exper imental  design  was  chosen  as  the  most  powerful 
available.  (Campbell  and  Stanley,  1963:34) 
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where:  X  is  the  treatment 

0  is  the  treatment  group  pretest 
1 

0  is  the  treatment  group  post-test 
2 

0  is  the  control  group  pretest 

3 

0  is  the  control  group  post-test 

4 

Figure  IS.  Experimental  Design 

Description  of  Quas  i lExperiment .  A  survey  instrument 
called  the  Quality  Questionnaire  was  used  to  measure 
attitudes  (Hayman  and  Schneider,  1989).  The  instrument 
consists  of  fifty-four  statements  which  allow  a  response  on 
a  seven  point  Likert  scale  ranging  from  strongly  disagree  to 
strongly  agree.  Because  the  presentation  of  the  scale 
includes  the  integers  one  through  seven,  the  data  obtained 
are  assumed  to  be  of  interval  level.  The  survey  yields 
results  that  allow  the  use  of  T-tests  and  other  parametric 
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statistical  tests  (Hayraan  and  Schneider,  1989:23).  The 
population  characteristics  of  the  sample  data  are  assumed 
to  be  normally  distributed  from  a  population  of  infinite 
size . 
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Figure  19.  The  Survey’s  Likert  Scale 

The  control  group  was  selected  to  match  experience  and 
authority  levels  with  the  treatment  group  as  closely  as 
possible.  Due  to  difficulty  in  selecting  members  in  similar 
positions  for  the  control  group,  the  pretest  was 
administered  to  the  control  group  two  months  after  the 
treatment  group  was  tested.  The  affect  of  this  delay  will 
be  discussed  below.  The  pretest  results  from  each  group 
were  compared  to  verify  that  in  fact  no  significant 
difference  in  scores  on  the  questionnaire  existed.  The 
quasi-experlmental  design  used  is  most  effective  when  the 
control  and  treatment  groups  are  equal  (Campbell  and 
Stanley,  1963:48).  This  was  done  using  a  two  sample  t-test 
with  the  null  hypothesis  being  no  difference  and  the 
alternate  hypothesis  being  any  difference  positive  or 
negative.  The  results  obtained  confirm  the  null  hypothesis; 
the  groups  are  declared  to  be  equal. 
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The  change  in  scores  on  each  statement  were  then 
compared  between  the  treatment  and  control  groups  using  a 
two-sample  paired  t-test.  The  null  hypothesis  was  that  no 
difference  in  change  in  scores  existed  between  the  groups. 
The  alternate  hypothesis  was  that  the  treatment  group  had  a 
significantly  different  change  in  scores  than  the  control 
group . 

Vall.dl.ty  of  the  Qyasi -Experiment .  The  non-equivalent 
control  group  design  controls  the  following  obstacles  to 
internal  validity:  history,  maturation,  testing,  and 
instrumentation.  The  control  of  maturation  and 
instrumentation  was  compromised  to  some  unknown  degree  due 
to  the  delayed  administration  of  the  pretest  to  the  control 
group  noted  earlier.  The  researcher  is  unaware  of  any 
significant  events  during  the  two  month  period  the  test  was 
delayed,  so  maturation  may  not  have  been  significantly 
affected.  Both  the  treatment  and  control  groups  had  a 
period  of  from  five  to  seven  months  between  their  pretest 
and  post-test,  so  the  difference  in  instrumentation  may  not 
be  important.  However,  this  weakness  in  the  administration 
of  the  experimental  design  is  noted,  with  its  corresponding 
potential  affect  on  the  experimental  results. 

The  obstacle  of  regression  is  controlled  if  the 
treatment  and  control  groups  are  equal.  A  test  of 
equivalency  was  performed.  The  obstacle  of  selection- 
maturation  interaction  is  reduced  when  the  members  of  the 


groups  are  assigned,  rather  than  volunteers.  The  obstacle 
of  mortality  was  controlled  by  assuring  that  no  members  of 
the  groups  dropped  out  between  the  pretest  and  post- test. 
(Campbell  and  Stanley,  1963:48,50) 

Table  7.  Validity  of  Quasi-Experiment 

The  non-equivalent  control  group 
experimental  design  controls  the 
following  obstacles  to  internal 
val idi ty . 

1)  History 

2)  Maturation 

3)  Testing 

4)  Instrumentation 

5)  Regression 

6)  Selection-Maturation 

7)  Mortality 

The  quasi-experiment  is  intended  to  augment  the  case 
analysis.  Its  external  validity,  or  ability  to  be 
generalized  beyond  the  tested  Groups,  is  not  a  critical 
issue.  The  case  analysis  itself  will  be  the  controlling 
factor  on  the  external  validity  of  the  results. 

Course  Development 

From  the  information  gathered  in  the  literature  review 
and  the  case  analysis,  a  training  course  was  developed  to 
implement  a  Performance  Management  Framework  in  the 
Engineering  Branch  of  Air  Force  Civil  Engineering  at  the 
base  level.  This  training  course  is  a  synthesis  of  the 
information  drawn  from  the  case  analysis  with  the 
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theoretical  information  collected  from  the  literature 
review.  The  method  used  to  develop  this  training  course 
consisted  of  the  following  nine  steps. 

1.  Identify  overall  goal  and  target  audience. 

2.  Identify  subject  areas  that  contribute  to  goal. 

3.  Identify  educational  objectives  within  each 
sub j  ect . 

4.  Structure  subjects  into  integrated  program.  Set 
time  allotment  for  each  subject. 

5.  Select  educational  tools/approach  to  communicate 
educational  objectives  for  each  subject. 

6.  Design  techniques  for  communicating  objectives. 

7.  Prepare  lesson  plans  for  each  technique. 

8.  Create  student  handbook  for  use  during  training. 

9.  Create  leaders  guide  book. 

Following  the  procedure  outlined  above,  a  Plan  of 
Instruction,  Student's  Handbook  and  Leader’s  Guide  were 
created  for  a  three  day  training  course.  The  course  is 
specifically  tailored  to  the  engineering  environment, 
considering  the  unique  needs  and  opportunities  observed  in 
the  case  analysis  and  the  literature  review.  The  course  is 
to  be  presented  at  the  squadron  location  to  the  engineers, 
supervisors,  and  other  individuals  involved  in  the 
preparation  of  contract  projects  in  the  Engineering  Branch. 
This  training  course  is  further  described  in  Chapter  V. 


Summary 

A  case  analysis,  supplemented  by  a  quasi -experiment  and 
a  literature  review,  were  chosen  to  answer  the  investigative 
questions.  The  case  analysis  provides  an  intensive 
examination  of  a  particular  subject,  and  therefore  adds  to 
the  knowledge  of  the  subject.  The  information  gained  from 
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the  case  analysis  was  compared  to  that  gathered  in  the 
literature  review.  Comprehensive  statements  of  trends  and 
characteristics  observed  were  then  made,  along  with 
appropriate  hypotheses  and  recommendations  for  further 
research.  A  training  course  and  handbook  to  guide 
implementation  of  performance  management  in  other 
engineering  organizations  were  developed.  The  course  is 
unique  in  that  it  focuses  on  the  particular  needs  of  the  Air 
Force  engineering  environment.  This  training  course  and 
handbook,  along  with  the  above  mentioned  conclusions, 
recommendations,  and  hypotheses,  are  presented  in  Chapter  V. 
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IV.  Results  and  Discussion 


Overview 

A  performance  management  team  was  formed  in  the 
Engineering  Branch  to  evaluate  and  improve  the  design  review 
process.  The  team  worked  through  a  problem  solving  routine 
for  the  Corps  of  Engineers  design  review.  This  routine 
actually  improved  the  process.  From  watching  this  team  in 
action,  the  researcher  observed  principles  of  performance 
management  in  an  Engineering  Branch  environment.  A  rigorous 
approach  was  used  to  record,  sort,  and  analyze  these 
observations . 

The  Quality  Questionnaire  survey  was  administered  to 
the  performance  management  team  and  a  control  group.  The 
change  in  attitude  toward  quality  was  compared  between  the 
two  groups.  Recommendations  will  be  made  and  conclusions 
drawn  from  these  results  in  Chapter  V. 

Case  Analysis 

o_i  Case.  The  Chief  of  the  Engineering 
Branch  of  the  2750  Civil  Engineering  Squadron  at  Wright- 
Patterson  Air  Force  Base,  Ohio  was  approached  during  August 
1989.  The  possibility  of  implementing  Total  Quality 
Management  (TQM)  in  the  branch  was  discussed.  A  team  was 
proposed  to  evaluate  the  performance  of  a  work  process  in 
the  Engineering  Branch. 


The  branch  Chief  was  skeptical  of  the  usefulness  of  TQM 
in  an  engineering  environment.  Previous  efforts  to  use  TQM 
in  the  Engineering  Branch  had  been  resisted  for  this  reason , 
though  other  branches  in  the  squadron  had  used  TQM  Process 
Action  Teams  (PATs)  and  Quality  Circles.  However,  th«.  Chief 
agreed  to  support  the  formation  of  a  team.  The  design 
review  process  was  suggested  as  a  focus.  The  branch  had 
been  having  difficulty  making  consistent  and  thorough 
comments  on  projects.  The  process  was  in  need  of  an 
overhaul.  The  Chief  felt  that  this  would  be  an  ideal 
opportunity  to  give  TQM  a  try.  A  plan  was  made  to  begin  the 
effort  after  the  start  of  the  fiscal  year,  01  October  1989. 

The  Engineering  Branch  Chief  hand-picked  a  PAT  to 
include  one  member  each  from  programming,  design,  design 
support,  and  construction  management.  The  supervisor  of  all 
these  members  was  included  as  well.  Each  member  was  chosen 

Table  8.  The  Five  Team  Members 

1.  Programming  Section  Representative 

2.  Design  Section  Representative 

3.  Design  Support  Representative 

4.  Construction  Management  Representative 

5.  Engineering  Branch  Supervisor 

only  on  the  basis  of  their  familiarity  with  the  function  of 
their  particular  sections  in  the  design  review  process. 

None  were  perceived  to  have  any  predisposition  toward 
quality  improvement  over  others  in  the  same  sections. 
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The  researcher  coordinated  the  preparation  for  the  PAT 
with  a  member  of  the  Industrial  Engineering  Branch  (DEI) . 
This  individual  had  experience  with  PATs  and  Quality  Circles 
in  other  branches  of  the  squadron.  A  plan  was  developed  to 
implement  the  PAT  in  the  Engineering  Branch.  The  researcher 
would  function  as  the  facilitator  for  the  team,  leading  the 
team  through  the  problem  solving  routine.  The  DEI  member 
would  provide  initial  training  and  attend  the  meetings  as  an 
observer  and  resource  person  as  needed. 

The  initial  team  meeting  was  held  02  Nov  89.  Training 
was  provided  to  introduce  the  problem  solving  routine  the 
team  would  follow.  This  routine  is  shown  in  Figure  20. 


Write  Flow  Chart 
for  Process 


1 

4 

k 


Identify  Problem  Causes 

4 

Develop  Solutions 


Implement  and 
Evaluate  Solutions 


Figure  20.  The  Problem  Solving  Routine 


The  problem  solving  routine  followed  was  based  on  the 
scientific  method.  Some  group  problem  solving  techniques 
the  team  might  find  useful  were  introduced  as  well.  These 
included  brainstorming,  cause  and  effect  analysis,  and 
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pareto  analysis.  Details  of  how  to  use  these  tools  were 
withheld  until  the  team  might  actually  have  need  for  them. 

After  the  training,  and  throughout  the  following 
meetings,  the  team  followed  the  problem  solving  routine  to 
evaluate  and  improve  the  design  review  processes.  Four 
separate  processes  of  design  review  were  identified, 
distinguished  by  the  different  design  agents  acting. 


Table  9.  The  Design  Review  Processes 


1.  Corps  of  Engineers 

2.  Operations  and  Maintenance 
Architect/ Engineer 

3.  Operations  and  Maintenance 
In-House 

4.  Aeronautical  Systems  Division 


The  process  flow  charts  for  each  of  these  four  processes 
were  the  first  thing  the  team  accomplished.  These  flow 
charts  are  displayed  in  Appendix  E. 


Table  10.  Corps  of  Engineers  Review  Process  Steps 


1.  Designer  Selected 

2.  Submittal  Arrives 

3.  Distribute  for  Review 

4.  Collect  and  Filter  Comments 

5.  Send  to  Headquarters 

6.  Repeat  from  Step  2  until  90*/. 

7.  End 
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The  team  selected  the  Corps  of  Engineers  process  as  the 
first  one  to  study.  They  decided  to  follow  the  rest  of  the 
steps  in  the  problem  solving  routine,  on  this  process  before 
beginning  on  the  next  process.  This  was  accomplished  in 
one-hour  weekly  meetings,  and  was  completed  01  Mar  90.  A 
summary  of  each  of  these  meetings  is  included  in  Appendix  D. 
An  integrated  list  of  the  problems  identified,  solutions 
proposed,  and  actions  recommended  for  implementation  for 
each  step  of  the  Corps  of  Engineers  design  review  process  is 
given  in  Appendix  F. 

The  team  continued  to  meet  after  01  Mar  90.  The 
Operations  and  Maintenance  design  review  process  was 
selected  for  attention  next.  The  researcher  continued  to 
meet  with  the  team  as  facilitator  through  29  May  90.  The 
study  of  the  second  process  was  not  completed  by  the  team  in 
time  to  be  included  in  this  research.  The  team’s  intention 
was  to  continue  meeting  until  all  four  processes  were 
completed  and  improvements  implemented  and  evaluated. 

Case .  As  a  consequence  of  their  evaluation 
of  the  Corps  of  Engineers  design  review  process,  the  team 
recommended  actions  which  were  implemented  to  improve  the 
process.  Measures  were  established  to  track  the  effect  of 
the  implementations  and  determine  if  in  fact  improvements 
were  realized.  In  the  short  amount  of  time  available  after 
the  actions  were  taken,  no  significant  data  from  these 
measures  were  obtained  to  document  the  effectiveness  of  the 
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improvements.  The  actiohs  taken  as  solutions  to  specific 
problems  can  be  reported,  however,  and  statements  made  of 
how  they  seemed  to  be  working.  These  actions  are  listed  in 
Table  1 1  . 


Table  11.  Solutions  Resulting  from  Case 

Provide  Base  Input  to  Selection  Committee 
Create  a  Plan  Review  Room 

Combine  and  Update  Review  Distribution  Lists 
Develop  a  User  Review  Checklist 
Make  Sure  CE  Shops  are  Included  in  Review 
Develop  Procedure  for  Environmental  Review 

One  problem  identified  was  that  the  Architect/Engineer 
design  firms  selected  by  the  Corps  of  Engineers  were 
consistently  giving  a  poor  response  to  projects  at  Wright- 
Patterson  Air  Force  Base.  This  problem  was  traced  to  the 
selection  of  the  designer  in  the  process.  The  cause  was 
determined  to  be  that  no  base  input  was  used  by  the  Corps  in 
selecting  designers.  Consequently,  the  design  firms  were 
often  located  in  Louisville,  KY,  where  the  Corps  office  was. 
Some  firms  whose  performance  was  consistently  bad  seemed  to 
keep  being  selected,  when  the  base  would  have  ruled  them  out 
of  contention  long  ago.  The  solution  proposed  was  to 
provide  feedback  from  the  base  about  the  firms  in  contention 
for  new  projects,  or  better  yet  to  get  a  base  representative 
to  the  committee  meeting  in  which  the  designers  were 
selected.  The  team  discovered  that  the  Corps  had  an  open 
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invitation  for‘an  Air  Force  representative  to  sit  on  the 
committee  as  a  voting  member,  but  no  one  had  been  attending! 


Figure  2T.  Selection  Committee  Solution 


The  solution  was  for  the  Engineering  Branch  to  budget  funds 
to  send  a  representative  to  each  selection  committee 
meeting.  Better  communication  with  the  Corps  was 
established,  too,  in  order  to  be  aware  of  when  a  meeting  is 
scheduled.  The  team  is  confident  that  this  input  to  the 
selection  process  will  improve  the  calibre  of  designers 
selected . 

Another  problem  identified  was  that  some  of  the 
reviewing  agencies  on  base  were  not  returning  comments  in 
time  to  forward  to  the  Corps.  This  problem  was  traced  to 
the  Distribute  for  Review  step  in  the  process.  The  cause 
was  determined  to  be  that  the  limited  number  of  copies  of 
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the  submittal  available  had  to  be  passed  around  between  the 
various  agencies.  Those  who  received  the  package  last  did 
not  have  enough  time  to  perform  a  thorough  review.  The 
solution  was  to  create  a  plan  room  at  the  Engineering  Branch 
location.  The  action  taken  was  to  set  aside  an  area  where 
plans  could  be  laid  out  for  review,  and  send  a  letter  to 
each  agency  for  each  submittal  to  come,  review  and  comment. 
The  response  was  good.  The  agencies  were  sending 
representatives  to  review  the  submittals  in  the  time  frame 
allowed.  More  and  better  comments  were  being  received, 
because  the  Engineering  Branch  representatives  were  right 
there  to  answer  any  questions  the  reviewers  might  have.  The 
branch  was  able  to  keep  a  record  and  keep  better  control  of 
who  was  reviewing  projects  as  a  result. 


Problem 

Comments  being 
returned  late 


Solution 

Create  a 
plan  room 


Action 

Invite  reviewers  to 
branch  location 


Figure  22.  Plan  Room  Solution 
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Figure  23.  Review  Distribution  List  Solution 


Conflicting  distribution  lists  for  review  was  another 
problem.  The  two  sections  responsible  for  different  levels 
of  the  design  review  had  different  lists  of  who  should 
review  the  project.  Further,  the  Chief  of  Design  and  Chief 
of  the  Engineering  Branch  were  on  neither  list.  This 
problem  fell  under  the  Distribute  for  Review  step  of  the 
process.  The  cause  was  lack  of  coordination  and  not 
updating  the  lists.  The  solution  was  to  make  one  list  for 
everyone’s  use.  The  action  taken  was  to  make  one  person  in 
the  Design  section  responsible  for  all  design  review.  This 
person  would  keep  the  master  distribution  list  and  update  it 
at  least  yearly.  All  letters  to  agencies  inviting  them  to 
review  submittals,  and  their  comments,  would  be  handled  by 
this  one  person.  The  team  is  confident  that  the  other 
agencies  will  appreciate  the  single  point  of  contact  for  the 
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branch,  and  the  elimination  of  duplicate  efforts  of  review. 

Another  problem  found  was  poor  comments  often  made  by 
the  users.  This  problem  was  identified  in  the  Distribute 
for  Review  step  of  the  process.  The  cause  was  that  the 
users  often  did  not  understand  the  construction  drawings  in 
the  submittals,  and  did  not  know  what  to  look  for.  The 


Figure  24.  User  Checklist  Solution 

solution  proposed  was  the  creation  of  a  checklist  to  prompt 
the  users  with  items  they  should  look  for  in  their  review. 
The  action  taken  was  to  assign  a  member  of  the  team  to  find 
out  what  kind  of  checklists  city  and  state  governments  or 
other  Air  Force  organizations  used,  and  adapt  these  as 
needed  to  the  requirements  at  Wright-Patterson .  The  team  is 
hopeful  that  given  this  guidance,  users  will  be  better 
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equipped  to  review  the  submittals  intelligently  and  make 
comments  about  the  project  scope  and  special  requirements 
that  are  needed  to  satisfy  these  users. 

The  next  problem  was  identified  in  the  Collect, 

Compile,  and  Filter  step  of  the  process.  Comments  from  the 
Operations  and  Maintenance  Branch  (DEM) ,  responsible  for 
maintenance  of  completed  construction,  were  often 
incomplete.  Its  cause  was  a  breakdown  within  DEM  in  getting 
all  the  various  disciplines  of  maintenance  to  review  the 


Figure  25.  Shops'  Design  Review  Solution 


project.  Just  one  shop,  for  example  the  plumbing  shop, 
would  review  the  submittal,  but  comments  from  the 
electrical,  HVAC ,  and  carpentry  shops  would  be  missing.  The 
solution  was  to  make  sure  the  projects  were  being 
coordinated  with  all  the  shops.  The  action  taken  was  to 
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suggest  to  DEM  that  a  review  meeting  be  held  for  each 
submittal.  A  representative  from  each  shop  would  gather  at 
one  location  with  the  project  engineer  and  all  comments 
would  be  discussed.  The  team  anticipates  that  this  offer 
will  result  in  better  communication  with  DEM  and  more 
thorough  comments. 

A  growing  problem  was  found  to  be  coordination  with  the 
Environmental  Management  (EM)  office.  With  the  increasing 
emphasis  on  environmental  concerns,  many  of  EM’s  functions 
were  new  and  unknown  to  the  Engineering  Branch.  Their 
comments  were  often  vague  and  general  in  nature.  This 
problem  was  traced  to  the  Collect,  Compile,  and  Filter  step 


Figure  26.  Environmental  Management  Review  Solution 

of  the  process.  The  solution  was  to  open  communication  with 
EM  and  gain  an  understanding  of  their  organization  and 
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purpose.  The  action  implemented  was  setting  up  a  session 
between  the  EM  Branch  chiefs  and  the  Engineering  Branch 
chiefs  to  get  acquainted.  Further  action  would  be  pursued 
from  there.  The  team  felt  that  opening  this  communication 
was  critical,  and  would  result  in  better  cooperation  between 
the  organizations. 

These  actions  illustrate  the  kind  of  improvements  the 
team  was  able  to  develop  during  their  study  of  the  Corps  of 
Engineers  design  review  process.  None  of  these  are 
revolutionary  or  remarkably  creative  in  and  of  themselves. 
What  is  remarkable  is  that  these  were  definite  actions  taken 
to  solve  specific  problems  the  branch  experienced.  In  some 
cases  the  problems  had  existed  for  years.  It  took  the 
formation  of  the  performance  management  team  and  their 
mandate  to  evaluate  and  improve  the  process  to  see  these 
problems  corrected. 

2b§®Evat ions  from  Case  Analysis.  The  goal  of  the  case 
observation  was  to  add  to  the  body  of  knowledge  on  the 
subject  of  performance  management  in  the  engineering 
environment.  The  results  the  team  achieved  were 
commendable ,  but  were  not  the  goal  of  this  research.  The 
experiences  of  the  team,  as  it  worked  through  the  problem 
solving  routine,  were  recorded  in  order  to  reveal  principles 
of  performance  management. 

2^5 Procedure .  The  procedure  followed  to 
withdraw  principles  from  the  case  analysis  was  to  record  and 
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synthesize  the  observations.  Appendix  D  contains  records 
summarizing  each  meeting  the  team  held  in  studying  the  Corps 
of  Engineers  design  review  process.  In  addition,  each 
record  contains  notes  made  by  the  researcher  of  observations 

Table  12.  Case  Analysis  Observation  Synthesis  Procedure 

1.  Attend  and  observe  meeting 

2.  Make  notes  of  observations 

3.  Assign  notes  to  subjects 

4.  Synthesize  principles  from  subjects 

pertaining  to  performance  management.  These  notes  are  the 
researcher’s  evaluations  and  judgments  of  the  experiences  of 
the  team.  After  these  notes,  each  entry  lists  several 
subjects  to  which  the  researcher  believed  the  contents  of 
the  notes  applied.  This  is  how  the  observations  were 
grouped  into  principles  of  performance  management.  These 
principles  are  discussed  under  the  next  heading. 


Table  13.  Case  Observation  Subjects 


1 .  Training 

2.  Implementation 

3.  Process  Technique 

4.  Goal  of  Improvement 

5.  Participation 

6.  Culture  Change 

7.  Facilitator 

8.  Management  Commitment 

9.  Management  Direction 

10.  Measurement 
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Discussion  of  Observations.  The  subjects 


identified  in  the  observations  of  the  case  are  listed  in 
Table  11.  These  subjects  each  appeared  in  notes  from  one  or 
more  of  the  meetings  summarized  in  Appendix  D.  Each  subject 
is  discussed  briefly  below  in  light  of  the  observations  made 
of  the  performance  management  team. 

Training  needs  to  be  appropriate  to  the  experience 
level  of  the  people  on  the  team.  Adequate  training  is 
important.  Skimping  on  training  at  the  start  may  result  in 
the  team  getting  stuck  later  on.  For  example,  if  the  team 
is  not  adequately  trained  in  the  group  technique  of 
brainstorming,  one  or  two  members  may  tend  to  dominate 
discussion.  This  would  tend  to  remove  important 
contributions  from  other  members  from  the  group’s  attention. 

The  problem  solving  routine  followed  (diagram  process 
flow,  identify  problems,  propose  solutions,  recommend 
actions  for  implementation,  establish  measures  of  success) 
proved  to  be  easy  to  use.  It  provided  the  framework  needed 
to  direct  the  team’ 3  attention  in  evaluating  the  process. 

The  biggest  shortcoming  in  its  use  was  an  inadequate  amount 
of  time  spent  identifying  problems.  The  team  needs  to  be 
challenged  to  think  more  deeply  and  trace  the  surface 
problems  they  recall  to  the  underlying  causes.  In  addition, 
the  process  might  be  easier  to  follow  if  the  goal  or 
objective  of  the  team  is  decided  upon  at  the  outset  as 
something  more  specific  than  Just  'improve  this  process.' 
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Table  14.  Implementation  Principles 


1.  Follow  problem  solving  routine 

2.  Establish  goal  at  outset 

3.  Dig  for  root  cause  of  problems 

4.  Invite  other  affected  organizations 

5.  Implement  a  simple  action  early 

Another  area  that  needs  emphasis  is  inviting  the  other 
organizations  affected  by  the  process  interfaces  to  attend 
and  participate  in  a  meeting.  This  increases  the  awareness, 
understanding,  and  communication  needed  to  institute 
effective  improvements.  Taking  immediate  action  to 
implement  one  or  some  of  the  simpler  improvements  is 
recommended.  This  permits  the  team  to  see  the  fruit  of 
their  labors. 

The  process  technique  was  instrumental  in  permitting  an 
understanding  of  the  work  the  team  was  studying.  Getting 
the  steps  of  the  process  mapped  out  right  off  the  bat 
contributed  to  the  team’s  ability  to  move  right  through  the 
problem  and  solution  phases.  The  process  steps  can  be 
modified  later  as  they  are  better  understood.  It  is 
important  that  the  team  not  set  the  boundaries  of  their 
process  too  narrowly.  The  process  can  be  better  understood 
by  reaching  across  the  interfaces  into  the  supplier  and  user 
steps  of  the  process.  'How  does  it  get  to  us?"  and  "Where 
does  it  go  from  here?"  are  two  good  questions  to  ask. 

The  unique  characteristics  of  the  engineering 
environment  affect  performance  management.  Engineers  are 
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trained  problem  solvers.  They  may  not  need  training  as 
detailed  as  some  other  functions.  Engineers  as 
professionals  are  typically  proud  of  the  quality  of  their 
work.  Suggestions  that  their  work  lacks  quality  or  needs 
more  quality  will  likely  be  met  with  resistance. 


Engineers 


Work 


Qual i ty 


Engineers  are  Committed 
to  Work  for  Quality 

Figure  27.  Engineers  and  Quality 

The  goal  of  improvement  may  be  set  by  management  or 
determined  by  the  team.  In  either  case,  the  goal  needs  to 
be  clearly  stated  at  the  outset  of  the  team’s  effort.  The 
goal  must  be  definite  and  verifiable.  For  example,  rather 
than  say  "Improve  the  design  review  process’  say  'Reduce  all 
change  orders  by  50/C." 

Participation  is  a  fundamental  concept  in  team  problem 
solving.  Protocol  must  be  followed  to  assure  that  all 
members  are  given  a  chance  to  contribute.  In  addition, 
tangents  of  interest  to  part  of  the  team  but  of  no  interest 
to  others  must  be  controlled,  or  some  members  will  lose 
interest.  Progress  toward  the  goal  must  also  be  maintained, 
again  discouraging  tangents.  The  reigns  must  not  be  held 
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too  tightly,  though.  When  the  team  is  animated  and 
discussion  is  active,  provide  some  leeway  to  see  if  the 
discussion  will  turn  into  something  fruitful  and  directed  to 
the  task  before  curtailing  the  interaction  to  get  back  on 
track . 

A  change  in  culture  is  making  the  adjustment  in  attitude 
needed  to  view  the  organization’s  work  in  terms  of  its 
processes  and  seek  to  improve  those  work  processes.  This 
culture  change  can  be  accomplished  ’on  the  job'  by  getting 
the  team  involved  in  the  process  oriented  problem  solving 
routine.  Their  perspectives  will  change  as  their  enthusiasm 
increases  for  improving  their  work.  Encourage  the  team  to 
share  with  their  co-workers  what  they  are  doing  on  the  team. 
Have  them  invite  guests  to  meetings.  Spread  the  news. 

Use  of  a  facilitator  to  aid  the  team  in  following  the 
problem  solving  routine  and  keeping  on  track  is  encouraged. 
The  facilitator  must  be  careful  not  to  dominate  the 
meetings.  In  fact,  this  person  might  even  be  almost 
invisible  when  the  team  is  working  well  together.  But  when 
the  team  has  a  problem  and  needs  help  or  gets  sidetracked, 
the  facilitator  can  bring  an  objective  voice  to  bear  and 
assist  the  team.  This  person  must  be  ready  to  refresh  the 
team’s  memory  of  where  they  were  and  what  they  were  doing 
after  any  recess  in  their  regular  meetings.  He/she  must 
also  assure  that  the  materials  and  room  needed  for  the 
meetings  are  in  order.  The  person  facilitating  the  meetings 
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needs  to  receive  training  in  group  dynamics  and  problem 
solving  techniques  prior  to  stepping  into  the  role.  This 
small  investment  will  be  well  repaid  in  improved  meetings 
(Todd,  1990) .  The  facilitator  must  be  ready  to  be  strong 
and  step  in  as  peacemaker  when  the  team  starts  battling  over 
opposing  views. 

Table  15.  Facilitator  Duties  and  Needs 

1.  Be  invisible  (except 
when  needed) 

2.  Be  ready  to  help 

3.  Prepare  for  team  meetings 

4.  Receive  training  in  team 
dynamics  and  problem  solving 

5.  Keep  the  peace 

Management  must  be  committed  to  the  efforts  of  the 
performance  management  team.  However,  management  must  not 
crowd  the  team.  Their  interest  in  how  things  are 
progressing  needs  to  be  satisfied  by  reports  from  the  team 
leader  or  facilitator.  The  temptation  to  crash  in  on  a 
meeting  must  be  avoided,  for  their  presence  will  curtail 
team  participation.  The  first  line  supervisor  of  the 
members  of  the  team  is  an  exception.  This  person  may  be 
very  beneficial  to  the  team.  Implementation  actions  and 
delegation  are  more  easily  accomplished  when  such  a  person 
is  a  team  member.  Management  needs  to  make  arrangements  for 
whatever  resources  the  team  requires,  and  to  invite 
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representatives  from  other  organizations  'interfacing  with 
the  team’s  process  to  attend  meetings  if  needed. 

Management  has  two  important  selection  roles  to  perform 
for  the  performance  management  effort.  The  process  for  the 
team  to  focus  on  must  be  chosen.  Care  must  be  taken  not  to 
overload  the  team  on  their  first  attempt.  Pick  a  simple 
process  as  a  team’s  pilot  project.  Let  them  cut  their  teeth 
on  it.  More  detailed  processes  can  follow.  The  members  of 
the  team  must  be  carefully  selected.  They  must  know  their 
Jobs  well  and  be  able  to  bring  that  knowledge  to  the  group. 


Involved 

Management 


Select  Process 
Action  Team 


Select  Process 
for  Improvement 


Figure  28.  Management  Selection  of  Process  and  Team 


A  first  line  supervisor  can  be  a  useful  member,  bringing 
lots  of  related  information.  The  members  do  not  all  have  to 
be  peers.  Further  management  direction  is  needed  if  there 
is  some  environmental  or  organizational  change  which  affects 
or  alters  the  process  the  team  is  working  on.  The  team  must 
be  kept  up  to  date  on  any  new  information  which  may  affect 
the i r  effort. 
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Measurement  of  success  of  the  actions  implemented  is  a 


step  that  must  not  be  overlooked.  It  is  this  measurement 
which  will  provide  the  clearest  indication  01  the  value  of 
the  improvements  made.  Management  will  be  interested 
ultimately  in  seeing  a  report  of  each  problem  identified, 
solution  proposed,  action  taken,  and  measured  results  ~ f 
that  action.  The  achievements  of  the  team  need  to  be 
organized  in  this  manner. 

Table  16.  Contents  of  Record  of  Achievements 

1.  The  problem  identified 

2.  The  cause  of  the  problem 

3  The  solution  proposed 

4.  The  action  taken 

5.  The  measured  results 

Quality  Survey 

Results.  All  five  of  the  members  of  the  treatment 
group  returned  both  the  pretest  and  the  post-test  survey. 
Only  four  of  the  five  members  of  the  control  group  returned 
both.  This  affects  the  experiment  by  reducing  the  already 
small  sample  size,  which  will  increase  the  magnitude  of 
difference  which  will  be  found  significant.  In  addition, 
some  members  of  the  control  group  left  responses  blank. 

This  has  the  same  effect  for  individual  questions.  The  data 
collected  from  these  surveys  is  presented  in  Appendix  Q. 

The  Quality  Questionnaire  survey  is  divided  into  six  major 
categories,  with  questions  in  each  to  measure  attitudes 
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specific  to  that  category.  The  responses  shown  in 
Appendices  H  through  M  are  divided  into  these  six 
categories,  which  are  listed  in  Table  17. 


Table  17.  Quality  Questionnaire  Categories 


I .  Leadersh i p 

II.  Strategic  Quality  Planning 

III.  Human  Resource  Management 

IV.  Quality  Assurance  of 
Products  and  Services 

V.  Quality  Results 

VI.  Customer  Satisfaction 


The  stat ' stical  tests  of  the  data  are  contained  in 
Appendix  M.  The  assumptions,  hypotheses  tested,  and 
procedure  used  for  testing  are  recorded.  Four  separate 
tests  were  performed.  These  are  listed  in  Table  18  The 


Table  18.  Statistical  Tests  Performed 


1 . 

Compar i son 

o  f 

Groups ’ 

PreTest 

Results 

2  . 

Compar i son 

o  f 

Groups 

PreTest 

Var i ance 

3  . 

Compar i son 

o  f 

Groups ’ 

Change 

Results 

4  . 

Compar i son 

o  f 

Groups ' 

Change 

Var i ance 

first  test  was  for  the  equivalence  of  the  control  and 
treatment  groups  at  the  time  of  the  pretest.  This 
equivalence  is  a  condition  of  declaring  the  obstacle  of 
regression  controlled  in  the  experiment.  The  results  of 
this  test  are  contained  in  Appendix  I.  For  each  question, 


the  average  response  of  the  treatment  group  was  compared  to 
the  average  response  of  the  control  group.  For  all  but  one 
question,  the  two  samples  were  found  to  be  the  same,  at  the 
precision  level  alpha  of  0.05.  The  treatment  and  control 
group  were  then  declared  equal  for  the  purposes  of  this 
experiment,  and  the  obstacle  of  regression  controlled. 

Figure  29  illustrates  the  comparison  of  average  responses  in 
each  major  category  between  the  control  and  treatment 
groups . 

The  second  test  was  for  the  equivalence  of  variance 
between  the  control  and  treatment  groups  at  the  time  of  the 
pretest.  This  equivalence  is  a  condition  of  the  first 
test,  allowing  the  parametric  statistical  test  to  be  used. 

The  results  of  this  test  are  contained  in  Appendix  J.  For 
each  question,  the  variance  of  the  responses  of  the 
treatment  group  was  compared  to  the  variance  of  the 
responses  of  the  control  group.  For  all  but  four  questions, 
the  two  samples  were  found  to  be  the  same,  at  the  precision 
level  alpha  of  0.05.  The  treatment  and  control  group  were 
then  declared  to  have  equal  variance  for  the  purposes  of 
this  experiment,  supporting  the  assumption  of  the  first  test. 

The  third  test  was  for  the  degree  of  change  between  the 
control  and  treatment  groups  from  the  pretest  to  the  post¬ 
test.  Measuring  this  degree  of  change  was  the  objective  of 
the  experiment.  The  results  of  this  test  are  contained  in 
Appendix  K.  For  each  question,  the  average  change  in 
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For  Each  Survey  Category 


Figure  29.  Group  Pre-Test  Comparison 
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response  of  the  treatment  group  was  compared  to  the  average 
change  in  response  of  the  control  group.  For  all  but  two 
questions,  the  two  samples  were  found  to  be  the  same,  at  the 
precision  level  alpha  of  0.05.  The  treatment  and  control 
group  were  then  declared  equal  for  the  purposes  of  this 
experiment.  No  difference  in  change  in  attitude  toward  the 
six  quality  categories  was  found  over  the  course  of  this 
case  analysis  and  experiment.  Figure  30  illustrates  the 
comparison  of  average  change  in  responses  in  each  major 
category  between  the  control  and  treatment  groups. 

The  forth  test  was  for  the  equivalence  of  variance 
between  the  control  and  treatment  groups’  change  in 
responses  associated  with  the  third  test.  This  equivalence 
is  a  condition  of  the  third  test,  allowing  the  parametric 
statistical  test  to  be  used.  The  results  of  this  test  are 
contained  in  Appendix  L.  For  each  question,  the  variance  of 
the  responses  of  the  treatment  group  was  compared  to  the 
variance  of  the  responses  of  the  control  group.  For  all  but 
eleven  questions,  the  two  samples  were  found  to  be  the  same, 
at  the  precision  level  alpha  of  0.05.  The  treatment  and 
control  groups  could  not  be  declared  to  have  equal  variance 
based  on  this  test.  However,  the  matter  was  not  pursued  to 
the  extent  of  performing  non-parametric  tests. 

Discussion.  No  difference  was  found  in  change  of 
attitude  toward  quality  between  the  treatment  and  control 
groups.  The  treatment  group  was  exposed  to  initial  training 
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L  E  HL  IV.  V.  VL  Overall 


Figure  30.  Survey  Results 
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in  process  improvement  techniques.  They  participated  in 
seven  months  of  meetings  using  these  techniques  to  improve 
the  design  review  process.  The  control  group  was  not 
exposed  to  any  unordinary  training  or  techniques.  And  yet 
when  given  the  post-test,  the  control  group  showed  a  change 
in  attitude  toward  the  quality  categories  little  different 
than  the  treatment  group.  Several  reasons  are  possible  for 
this  lack  of  difference. 

Table  19.  Possible  Reasons  for  Lack  of  Difference 

1 .  Time  between  pretest  and  post-test 

2.  Inadequate  culture  change  training 

3.  Participation  doe3  not  change  attitude 

The  period  of  time  between  the  pretest  and  post-test 
may  not  have  been  long  enough  to  observe  significant 
attitude  changes  In  the  treatment  group.  Performance 
management  has  been  described  earlier  as  needing  a  long  term 
commitment.  Short  term  results  are  not  to  be  expected.  The 
seven  months  between  the  pretest  and  the  post-test  may  have 
been  insufficient  for  the  culture  change  that  accompanies 
process  improvement  efforts  to  be  seen.  Culture  change  is 
defined  as  a  transformation  of  viewpoint  from  'business  as 
usual'  to  ‘continuous  improvements."  If  more  time  had  been 
available  between  the  two  tests,  perhaps  a  significant 
difference  between  the  two  groups  would  have  been  observed. 


This  possibility  could  be  tested  for  by  administering 
another  post-test  at  some  later  date. 

Another  possible  reason  is  that  the  initial  training 
provided  to  the  treatment  group  did  not  include  adequate 
emphasis  on  culture  change.  The  training  provided  was 
completed  in  twenty  minutes  at  the  first  team  meeting.  An 
introduction  to  process  flow-charting  and  brainstorming 
techniques  was  all  that  was  included.  Perhaps  more  of  an 
emphasis  needed  to  be  made  on  change  of  attitude  to  one  that 
is  most  conducive  to  performance  management.  Without  this 
emphasis,  the  treatment  group  may  not  have  had  sufficient 
exposure  to  the  broad  spectrum  of  quality  issues  to  result 
in  changed  attitudes.  This  could  be  tested  for  by  testing  a 
similar  implementation  of  performance  management  in  another 
organization  where  initial  training  includes  culture  change 
training . 

An  additional  potential  reason  for  the  lack  of 
difference  in  attitude  change  is  that  the  performance 
management  framework  followed  does  not  need  or  produce 
attitude  change.  Engineers  by  profession  are  trained  in 
problem  solving.  The  subjects  of  this  case  analysis  and 
experiment  have,  as  engineers,  been  taught  to  seek  quality 
and  better  ways  to  solve  problems.  It  may  be  that  the  need 
for  a  cultural  change  prior  to  successful  performance 
management  in  an  engineering  environment  is  not  critical. 
However,  a  generalization  of  this  nature  cannot  be  made  on 
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the  basis  of  the  results  of  this  experiment  alone. 
Significant  additional  testing  of  other  engineering 
applications  of  performance  management  would  be  required. 

Summary 

A  performance  management  team  was  used  to  evaluate 
and  improve  the  Corps  of  Engineers  design  review  process. 
Improvements  that  resulted  from  this  team  effort  are 
summarized  in  Table  20.  The  observations  of  the  experiences 
of  the  team  were  catalogued  and  assigned  to  subjects. 
Principles  were  then  synthesized  from  these  observations. 

Table  20.  Results  of  Performance  Management  Team 

1.  Representation  on  Selection  Committee 

2.  Creation  of  Plan  Room  for  Review 

3.  Consolidate  the  Branch  Design  Review 
Point  of  Contact 

4.  Prepare  User  Review  Checklist 

5.  Suggest  Review  Meeting  with  DEM  Shops 

6.  Open  Communications  with  Environmental 
Management 

These  principles  were  compared  to  the  information  gathered 
in  the  review  of  literature.  From  this  comparison, 
recommendations  will  be  made  in  the  next  chapter. 
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V.  Conclusions  and  Recommendaations 


Overview 

This  chapter  presents  recommendations  drawn  from  the 
research  reported  in  Chapters  II  and  IV.  These 
recommendations  include  suggestions  for  more  effective 
implementation  of  performance  management.  In  addition, 
hypotheses  to  be  tested  in  future  research  are  proposed. 
Finally,  the  conclusion  of  this  research  is  stated  and 
supported . 

Recommendations 

Name  of  1^  mg  rove  me  n  t  E^f^ort.  Much  of  the  activity  in 
the  area  of  performance  evaluation  and  improvement  has  gone 
under  the  name  of  Total  Quality  Management  (TQM).  The 
Department  of  Defense  used  this  name  in  directing  the 
Services  to  adopt  a  framework  for  the  management  of 
performance  (Department,  1989b).  The  name  used  in  this 
thesis  has  been  "Performance  Management  Framework."  While  a 
name  does  not  a  program  make,  it  is  important  to  note  the 
reaction  of  personnel  in  Air  Force  Civil  Engineering  to  the 
name  TQM.  Statistical  process  control  and  X-bar  charts 
often  come  to  the  minds  of  people  familiar  with  TQM  when 
they  hear  the  name.  These  statistical  tools  are  important  to 
TQM  in  the  manufacturing  environment.  The  usefulness  of 
these  tools  in  the  design  environment  has  not  been  as  well 
established.  To  give  the  performance  management  effort  a 
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chance  to  get  off  the  ground,  a  name  without  negative 
preconceptions  to  engineers  needs  to  be  selected. 

The  Air  Force  Civil  Engineering  and  Services  community 
has  decided  to  use  the  name  'Engineering  and  Services 
Quality  Management  Program"  (Ahearn,  1990).  This  identifies 
the  effort  with  the  engineering  community.  It  also 
distinguishes  the  effort  from  stock  TQM.  Maintaining  this 
distinction  is  recommended. 


ENGINEERING  AND  SERVICES 
QUALITY  MANAGEMENT  PROGRAM 


Figure  31.  A  Name  for  the  Engineering  Improvement  Effort 

Use  of  Training  Course.  The  Performance  Management 
Framework  Training  Course  developed  in  this  research  and 
presented  in  Appendices  A,  B,  and  C  is  uniquely  suited  for 
use  in  the  base  level  Engineering  Branch  environment.  While 
dozens  of  guides  for  TQM  exist,  and  even  several  aimed  at 
service  organizations,  none  are  tailored  to  Air  Force  Civil 
Engineering  as  this  one  is. 

This  training  course  is  designed  to  assist  Engineering 
Branch  managers,  engineers,  and  others  who  contribute  to  the 
preparation  and  execution  of  projects  for  real  property 
construction,  maintenance,  and  repair.  These  individuals 
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are  challenged  with  providing  and  maintaining  quality 
facilities  for  Air  Force  organizations  to  perform  their 
missions.  Evaluation,  management  and  improvement  of  the 
performance  of  Engineering  Branch  personnel  is  central  to 
continuing  to  fulfill  this  mandate. 

Table  21.  Performance  Management  Framework  Objectives 

Foster  a  management  perspective  conducive  to 
performance  improvement 

Create  an  organizational  environment  promoting 
performance  improvement 

Continually  encourage  the  quest  to  discover  and 
implement  ways  to  improve  performance 

The  objectives  for  use  of  a  framework  for  performance 
management  are  listed  in  Table  21.  The  objectives  of  the 
training  course  which  presents  such  a  framework  are  listed 
in  Table  22. 


Table  22.  Training  Course  Objectives 


To  equip  civil  engineering  personnel  to  manage  the 
Performance  Management  Framework 

To  create  momentum  for  performance  improvement 

To  promote  within  civil  engineering  personnel  a 
perspective  for  improvement 

To  provide  the  essential  tools  and  techniques  to 
establish  and  maintain  a  Performance  Management 
Framework 
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The  goal  of  this  training  and  the  handbook  which 
accompanies  it  is  to  acquaint  Engineering  Branch  members 
with  a  framework  for  managing  performance  and  to  provide  the 
tools  needed  to  use  this  framework.  It  was  designed  to  be 
taught  by  a  member  of  the  Engineering  Branch.  This  person 
does  not  need  an  extensive  background  in  performance 
management  nor  experience  in  teaching.  A  few  hours  time  in 
advance  to  become  familiar  with  the  contents  of  the  course 
are  all  that  are  required.  However,  a  course  leader  with 
background  or  experience  in  either  of  these  would  likely  be 
more  effective  than  a  novice. 

The  training  course  is  designed  to  be  three  days  in 
length,  eight  hours  each  day  with  an  hour  for  lunch.  The 
material  is  presented  through  a  mixture  of  group  interaction 
and  exercises  that  encourage  experiencing  the  principles  as 
they  are  discovered.  The  course  is  to  be  hosted  at  the 
squadron  location,  to  allow  the  Engineering  Branch  members 
to  learn  in  their  own  environment,  and  to  minimize 
disruption  to  their  routine. 

This  course  was  developed  based  on  experiences  and 
lessons  learned  in  the  implementation  and  use  of  a  framework 
for  performance  management  in  an  actual  Engineering  Branch 
environment.  Air  Force  engineering  organizations  may  use 
thi3  course  in  conjuction  with  other  training  available  from 
the  Air  Force  Civil  Engineering  and  Services  Center  to  begin 
a  squadron-wide  performance  management  effort.  Contact  Ray 
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Hatch.  AFESC/DEMG ,  Tyndall  AFB ,  Florida,  AV  523-6401  for 
further  information  on  other  training  materials. 

The  course  presented  in  this  research  is  available  at 
little  cost  to  the  interested  organization.  No  teams  of 
consultants  are  needed.  The  course  may  be  taught  with 
materials  and  personnel  already  present  in  the  Engineering 
Branch.  It  provides  the  tools,  place  to  get  started  and 
attitude  for  improvement  that  are  needed  to  begin  improving 
the  way  work  is  accomplished  in  the  Engineering  Branch  at 
base  level . 

Several  characteristics  distinguish  the  training 
provided  by  this  course  from  other  training  available. 

These  are  the  reasons  this  training  course  might  be  chosen 
over  other  alternative  ways  to  implement  a  performance 
management  framework  in  civil  engineering.  First,  this 
course  recognizes  the  unique  nature  of  the  engineering 
environment.  The  fact  that  the  engineer’s  product  is  ideas 
and  designs  is  accounted  for.  Second ,  the  crucial 
importance  of  management  commitment  and  direction  is 
emphasized.  Management  involvement  from  the  outset  is 
critical  to  assuring  that  lasting  improvements  are  made 
(Edosomwan,  1987:67-8).  Third,  the  course  stresses  creative 
ways  of  measuring  the  output  of  engineering  which  is  often 
difficult  to  quantify.  For  performance  management  to  be 
effective,  a  system  of  measurement  is  essential  for 
monitoring  the  work  processes  before  and  during  the 
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improvement  efforts  (Dingus  and  Hrivnak,  1988:30).  Fourth, 
the  target  of  improvement  is  defined  as  more  than  just  the 
quality  issues  which  are  stressed  in  other  training.  The 
broader  goal  of  performance,  including  quality  but  also 
productivity,  effectiveness  and  efficiency  needs  to  be  the 
target  in  civil  engineering  (Armentrout,  1986:142-144). 

Fifth,  the  course  develops  the  issue  of  training,  to  provide 
individuals  the  tools  and  attitude  needed  to  participate  with 
others  in  improving  their  work.  Training  is  an  element  of 
performance  management  which  cannot  be  neglected,  for  both 
the  team  members  and  the  leaders  and  facilitators  (Todd, 
1990).  Finally,  the  innovative  process  viewpoint  on  work 
allows  the  activities  of  the  workplace  to  be  understood,  and 
consequently  improvement  proposed  that  get  to  the  root  of 
the  problem  (Kacker,  1988:41). 

Table  23.  Advantages  of  This  Training  Course 

1.  Recognizes  the  unique  nature  of  civil  engineering 

2.  Emphasizes  management  support  and  direction 

3.  Stresses  ways  to  measure  engineering  performance 

4.  Defines  broad  goal  of  improvement  beyond  quality 

5.  Develops  training  objectives  for  tools  and  attitude 

6.  Presents  process  viewpoint  of  work  activities 

I  mg  rove  me  nt  Teams.  A  fundamental  concept 
of  performance  management  is  participation.  This  means 
getting  the  workers  involved  in  evaluating  and  improving 
their  Jobs.  These  people  are  the  ones  most  familiar  with 


their  work  and  the  problems  they  face,  and  so  are  often  more 
aware  of  what  improvements  are  possible  than  management  is. 
Using  a  team  approach  also  encourages  more  creative 
solutions  to  problems,  as  the  group  can  bounce  thoughts 
around  to  come  up  with  new  ideas  (Todd,  1990). 

A  team  can  formed  all  from  the  same  functional  area,  or 
represent  a  functional  cross-section,  as  appropriate  for  the 
work  process  being  addressed.  From  five  to  eight  members 
are  ideal.  These  members  might  be  volunteers  or  hand-picked 
by  management  to  bring  the  needed  experience  and  knowledge 
to  the  team.  The  team  will  meet  together  over  a  period  of 
time,  usually  once  a  week,  to  collectively  study  and  propose 
solutions  to  a  work  process  or  problem  the  organization  has 
identified.  When  their  recommendations  are  complete,  their 
proposals  are  presented  to  management.  Management  must  then 
demonstrate  their  support  and  commitment  to  the  effort  by 
implementing  actions  recommended  by  the  team,  and 
recognizing  the  team's  efforts  through  awards  and  publicity. 
The  team  is  then  disbanded,  their  job  complete,  or  assigned 
a  new  process  or  problem  (Todd,  1990). 

Scope  of  Pil.pt  Project.  The  basic  pattern  for 
performance  management  is  to  establish  a  performance 
improvement  team  to  propose  and  implement  improvements  in  a 
work  process.  A  temptation  will  be  to  start  on  the  most 
problem-plagued,  complex  work  process  the  organization  has. 
This  can  lead  to  discouragement  of  the  team  and  failure  of 
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the  improvement  effort.  The  newly  formed  performance 
improvement  team  should  be  allowed  to  focus  on  a  less 
complicated  but  still  important  process  to  begin  with.  This 
is  often  called  a  pilot  project.  With  this  experience  under 
their  belt,  the  team,  or  even  another  team,  is  better  able 
to  proceed  on  to  more  difficult  challenges. 

?ZE°i*l®ses  for  Further  Research.  This  research  effort 
was  qualitative  in  nature.  As  such,  it  sought  to  increase 
the  body  of  knowledge  in  existence  on  the  subject  of 
performance  management  in  civil  engineering.  A  stated  goal 
of  this  research  was  to  develop  hypotheses  for  further 
research.  The  nature  of  this  subject  was  not  well  enough 
understood  to  propose  any  reasonable  hypotheses  to  be  tested 
in  this  research.  Having  developed  the  subject  through  this 
research,  several  hypotheses  can  now  be  proposed. 

Hypothesis:  Using  the  training  course  presented 

herein  will  result  in  saving  more  resources  than  are 
expended  in  the  first  year.  This  would  be  tested  by 
selecting  a  sample  population  of  base  level  Engineering 
Branches  and  using  the  course.  The  manpower  and  material 
resources  needed  for  training  and  implementation  would  be 
recorded.  The  resources  saved  due  to  improved  performance 
would  be  determined  by  the  measurement  systems  created  as  a 
result  of  the  performance  management  framework  itself.  The 
resources  expended  would  be  compared  to  those  saved  to  test 
whether  the  hypothesis  is  supported. 


84 


Performance  Management  Framework 


Figure  32.  Training  Course  Hypothesis 


Hypothesis:  The  name  Total  Quality  Management  carries 

an  unfavorable  impression  to  engineering  managers.  Related 
hypothesis:  The  extent  of  the  unfavorable  impression 

created  by  the  name  TQM  is  inversely  proportional  to  the 
individual's  degree  of  knowledge  of  TQM.  These  could  be 
tested  using  a  survey  instrument.  The  survey  might  a3k  for 
responses  to  statements  of  the  usefulness  of  various 
principles  of  performance  management.  Some  of  the 
statements  would  include  the  name  TQM  while  others  would 
not.  The  difference  in  response  between  those  with  the  name 
and  those  without  could  be  measured.  The  survey  could  also 
include  some  questions  to  determine  the  individuals 
knowledge  and  exposure  to  TQM.  These  results  could  be 
correlated  with  any  difference  in  response  to  the  previous 
statements . 

Hypothesis:  The  degree  of  perceived  management  support 

among  the  performance  management  team  members  will  correlate 
to  the  team’s  effectiveness.  This  could  be  tested  by  using 
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a  short  survey  instrument  to  measure  the  team  members 
impression  of  how  well  management  supported  their  efforts. 
The  resources  saved  resulting  from  the  team’s  improvement 
recommendations  could  then  be  determined  using  the  measures 
proposed  by  the  team  in  the  performance  management 
framework.  A  correlational  statistical  study  could  be 
oerformed  to  determine  if  in  fact  the  hypothesis  is 
supported.  Regression  could  be  used  to  develop  the 
relationship  between  management  support  and  effectiveness, 
as  illustrated  in  Figure  33. 


$ 

Saved 


Degree  of  Management  Support 
Figure  33.  Management  Support  Hypothesis 

Hypothesis:  Engineers  have  a  higher  initial 

appreciation  and  understanding  of  quality,  and  commitment  to 
quality  work,  than  other  career  fields.  This  results  in 
leS3  cultural  change  training  being  required  with  engineers 
than  with  others.  One  goal  of  cultural  change  training  is 
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to  alter  individuals  attitudes  toward  quality  and  convince 
them  of  the  importance  of  quality  improvements.  This  could 
be  tested  by  comparing  the  effectiveness  of  performance 
management  in  an  engineering  environment  with  its 
effectiveness  in  other  settings.  Various  degrees  of 
cultural  change  training  could  be  provided  in  each  case. 
Measures  would  need  to  be  established  for  quantifying  the 
effectiveness  of  performance  management. 

Hypothesis:  Change  in  attitude  toward  quality  will 

accompany  successful  implementation  of  performance 
management  in  an  organization.  This  change  in  attitude  will 
become  more  pronounced  as  time  progresses  from  the  date  of 
the  implementation.  This  could  be  tested  by  continuing  to 
administer  the  Quality  Questionnaire  (Hayman  and  Schneider, 
1989)  to  a  treat oent  and  control  group  at  regular  intervals 
after  implementation  in  the  treatment  group.  Regression 
analysis  could  be  used  to  measure  the  trend,  if  any,  in 
attitude  change  for  each  group  and  compare  the  trends. 
Answers  to  Investigative  Questions 

In  the  training  course  developed  and  the 
recommendations  listed  above,  the  investigative  questions 
have  been  answered.  Following  is  a  brief  summary. 

How  has  TQM  been  utilized  in  other  similar 
organizations?  What  benefits  have  been  achieved?  What 
changes  need  to  be  made  to  adapt  TQM  to  the  Contract 
Engineering  Branch?  TQM  has  been  used  extensively  in  the 
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manufacturing  environment,  especially  in  Japan.  Attention 
is  now  being  paid  to  using  the  same  principles  in  the  service 
or  government  environment.  The  TQM  tools  and  attitude  need 
to  be  adapted  to  civil  engineering  by  accounting  for  the 
unique  nature  of  civil  engineering,  emphasizing  management 
support  and  direction,  and  stressing  ways  to  measure 
engineering  performance.  The  broad  goal  of  improvement 
needs  to  be  defined  as  performance,  which  encompasses  more 
than  just  quality.  Training  objectives  for  the  tools  and 
attitude  required  must  be  developed,  and  the  process 
viewpoint  of  work  activities  presented. 

What  opportunities  and  needs  exist  within  the 

Branch  for  improvement?  The  work  activities  of 
an  Engineering  Branch  can  be  assigned  to  the  processes  to 
which  they  contribute.  Any  of  these  processes  can  be 
evaluated  and  improved  using  performance  management.  The 
design  review  process  is  just  one  example  analyzed  in  this 
case.  Some  potential  processes  for  improvement  using  a 
performance  improvement  framework  are  listed  in  Table  24. 

Table  24.  Engineering  Branch  Processes 

Design  Review 
Contract  Management 
Project  Programming 
Change  Order  Management 
Accomplishment  of 
In-house  Design 
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What  training  of  personnel  i_s  used  prior  to  or 
during  implementation  of  TQM?  Training  must  include 
emphasis  on  both  tools  and  attitude.  Cultural  change 
training  seeks  to  alter  the  viewpoint  of  the  employees  and 
prepare  them  to  seek  improvements.  Training  in  specific 
group  problem  solving  tools  is  needed  to  allow  the  group  to 
participate  as  a  team  in  evaluating  and  improving  their  work 
processes.  This  training  needs  to  be  targeted  at  the  people 
most  likely  to  use  it,  and  not  just  given  to  all 
indiscriminately . 

What  obstacles  are  encountered  during  the  initial 
i t,§! A 5D  of  TQM?  An  obstacle  encountered  in  the 
engineering  environment  are  the  project  rather  than  product 
orientation  of  engineering.  Another  is  the  frequently 
changing  requirements  that  users  place  on  the  Engineering 
Branch.  A  third  is  the  difficulty  in  measuring  the  output 
of  an  engineering  organization.  Inadequate  management 
commitment  and  people’s  prejudices  against  change  are  other 
potential  obstacles. 

What  short  range  benefits  can  be  measured  after 
i!!}EA®5?®!Qtation  9l  TQM?  Short  range  benefits  include  gaining 
a  better  understanding  of  the  work  activities  in 
organizations.  Some  obvious  improvements  can  be  made 
Immediately  upon  discovering  them  in  the  evaluation  process 
as  well.  These  benefits  can  be  measured  in  decreased  time 
spent  in  wasted  energy. 
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What  outside  resources  gr i_vate  contractors  or 

procured  training)  are  needed  to  facilitate  the  performance 
management  effort?  No  outside  resources  are  required  for 
training  or  facilitation.  The  training  course  developed  in 
this  effort  can  be  used  at  little  cost  and  with  none  other 
than  the  personnel  available  to  the  squadron. 

How  do  attitudes  toward  quality  of  work  change  among 
emg^o^ees  involved  in  gerformance  management?  In  the  short 
term,  six  months,  no  change  in  attitude  was  measured  in  the 
case  analyzed  in  this  work.  Over  a  longer  time  interval 
such  changes  might  become  apparent. 

How  is  a  performance  management  framework  used  to  guide 
improvement  efforts  in  the  particular  situation  of  an 
Engineering  Branch?  Performance  management  can  be  used 
quite  effectively  in  an  Engineering  Branch.  First,  top 
management  must  be  committed  to  support  and  direct  the 
effort.  They  must  provide  training  to  the  people  who  will 
be  involved.  Management  must  further  pick  a  work  process  to 
be  evaluated  and  improved  and  choose  a  team  to  look  at  it. 
This  team  must  follow  a  structured  procedure  for  problem 
solving.  They  must  understand  the  work  process  involved, 
and  the  steps  it  contains.  The  team  then  identifies 
problems  which  plague  this  process,  and  proposes  solutions 
to  the^a  problems.  Finally,  the  team  recommends  actions  for 
implementation  to  management.  The  cycle  is  then  completed 
when  management  approves  improvement  changes  be  made.  The 
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participants  in  the  effort  are  rewarded,  and  the 
organization  is  ready  to  begin  again  with  a  new  team  and 
process.  A3  performance  management  becomes  more  familiar, 
several  of  these  cycles  might  be  going  on  simultaneously. 


Management  Initiates  and 
Directs  Improvement  Effort 


V 

Participative  Employee 
Performance  Improvement  Team 
Evaluates  Work  Process 


Actions  for  Improvements 
are  Recommended  to  Management 


Management  Approves  Changes 
and  Rewards  Participants 


Figure  o*± .  Performance  Management  Framework 


What  guidance  do  Engineering  Branch  managers  need  to 
design  and  implement  a  performance  management  framework  in 
the i_r  organizations?  Managers  need  specific  guidance  in  how 
to  implement  performance  management.  This  guidance  includes 
who  and  how  to  train,  the  importance  of  management  support 
and  direction,  and  a  way  to  get  an  improvement  effort 
started.  The  guidance  for  an  Engineering  Branch  manager 
needs  to  consider  the  particular  characteristics  of  the  Air 
Force  engineering  environment.  If  guidance  is  used  that  was 
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designed  for  operational  organizations  on  the  Air  Force 


base,  the  effort  may  not  be  successful.  The  training  course 
presented  here  is  tailored  specifically  to  the  engineering 
environment,  provided  the  guidance  the  Engineering  Branch 
manager  needs. 


Table  25.  Guidance  Needed  by  Engineering  Branch  Managers 


1.  Who  and  how  to  train 

2.  Importance  of  management 

direction  and  support 

3.  Starting  point  for  the 

improvement  effort 


Performance  management  is  a  powerful  concept  which 
promises  substantial  Increases  in  the  quality  and 
effectiveness  of  an  organization’s  work,  together  with 
decreases  in  costs.  These  gains  are  accomplished  by  using 
employee  work  teams  to  evaluate  work  processes  and  eliminate 
waste  and  errors.  The  concept  has  proven  successful  in  a 
number  of  work  situations,  both  manufacturing  and  service, 
public  and  private. 

The  Engineering  Branch  in  a  base  level  Civil 
Engineering  Squadron  can  benefit  from  performance 
management.  The  work  processes  which  make  up  the  activity 
of  the  Engineering  Branch  are  suited  for  analysis  using  the 
process  improvement  techniques.  The  engineers  themselves 
are  already  trained  problem  solvers.  They  just  need  to  have 
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their  talents  directed  toward  improving  the  way  they  work. 
This  direction  is  provided  by  the  performance  management 
framework . 

A  need  for  a  framework  for  performance  management  for 
the  base  level  Engineering  Branch  motivated  this  research. 
The  opportunities  and  desire  existed  to  improve  the 
organization’s  work  processes.  Lacking  were  the  attitude, 
t oo 1 3  ,  and  starting  point  needed  to  create  a  successful 
improvement  effort.  This  research  followed  and  documented 
the  use  of  performance  management  in  an  Engineering  Branch 
environment . 

A  performance  management  team  was  implemented  at  the 
Engineering  Branch  of  the  2750  CES  at  Wr i ght-Patterson  Air 
Force  Base.  The  team  focused  on  the  process  of  reviewing 
projects  designed  by  Architect-Engineers  and  the  Corps  of 
Engineers.  As  a  result  of  this  team  effort,  improvements  to 
the  process  were  made.  More  timely  and  better  comments  were 
being  provided  to  the  designers  after  the  improvement 
actions . 

From  the  experiences  gained  in  the  analysis  of  this 
case,  a  three  day  training  course  was  developed.  This 
course  was  created  to  provide  the  tools  and  attitude  needed 
to  begin  performance  management  efforts  in  other  Engineering 
Branches.  The  course  was  designed  to  provide  a  starting 
point,  and  is  a  guide  to  the  performance  management  effort. 
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Aggendi x  A^_  Plan  of  Instruction 
This  appendix  contains  the  Plan  of  Instruction  for  a 
three  day  training  course  in  performance  management.  The 
Student’s  Handbook  and  Leader’s  Guide  for  the  course  are 
given  in  Appendix  B  and  Appendix  C  respectively. 

The  objectives,  schedule  and  scope  are  presented  here. 
A  general  lesson  plan  is  given  for  each  of  the  eighteen 
lessons.  Each  lesson  plan  includes  objectives  and  an 
outline  of  the  subjects  discussed  in  that  lesson. 
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PLAN  OF  INSTRUCTION 


Course  Title: 

Objective: 

Audience : 

Scope : 

Materials : 


Civil  Engineering  Performance  Management 
Framework  Training 

To  motivate  and  equip  supervisors  and  project 
management  and  support  personnel  in  the 
Engineering  Branch  of  Air  Force  Civil 
Engineering  Squadrons  to  implement  the 
Performance  Management  Framework. 

This  course  is  designed  to  be  presented  to 
the  Chief  of  Engineering,  Chief  of  Design, 
Chief  of  Construction  management,  Chief  of 
Programming,  Chief  of  Drafting,  and 
engineers,  secretaries,  technicians  and  other 
members  of  the  Engineering  Branch  as  desired. 
Group  size  of  seven  to  fifteen  members  is 
recommended . 

The  course  is  designed  to  last  three  days, 
with  seven  hours  of  training  each  day.  Each 
hour  should  include  a  five  to  ten  minute 
break.  The  leader’s  guide  provides  enough 
information  for  a  member  of  the  Engineering 
Branch  to  conduct  the  training  with  nominal 
preparation.  No  formal  leader  training  is 
required . 

One  leader’s  guide  and  sufficient  student’s 
guides  for  all  the  members  of  the  training 
group  are  required.  Additionally,  an 
overhead  projector  and  clean  transparency 
film,  or  a  flip  chart  with  clean  pad  of 
paper,  along  with  colored  pens  are  needed. 
Additional  material  requirements  are  noted  in 
specific  lessons. 
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Contents:  Lesson 

Description 

Time 

Dav  1 

01 

Introducti on/Di stinctives 

1 

hour 

02 

Benefits  Available 

1 

hour 

03 

Management  Commitment 

1 

hour 

04 

Management  Support 

1 

hour 

Lunch 

1 

hour 

05 

Team  Participation 

1 

hour 

06 

Participation  Exercise 

2 

hours 

Day  2 

07 

Training  Objectives 

1 

hour 

08 

Training  Techniques 

1 

hour 

09 

Process  Improvement 

1 

hour 

10 

Process  Flow  Exercise 

1 

hour 

Lunch 

1 

hour 

11 

Process  Flow-charting 

1 

hour 

12 

Performance  Definition 

1 

hour 

13 

Performance  Components 

1 

hour 

Day  3 

14 

Implementation  Details 

1 

hour 

15 

Implementation  Exercise 

3 

hours 

Lunch 

1 

hour 

16 

Measurement  Exercise 

1 

hour 

17 

Cause  and  Effect  Discussion 

1 

hour 

18 

Objectives  Matrix 

1 

hour 
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Lesson  01  Introduction/Engineering  Environment 

Distincti ves 

Time:  1  hour 

Objectives:  Distinguish  the  Performance  Management 

Framework  from  other  management  fads  and 
gimmicks  which  have  come  and  gone. 

Communicate  that  this  program  is  one  which 
civil  engineering  personnel  design  themselves 
to  fit  their  own  particular  situations. 

Technique:  Group  brainstorming  -  Have  group  generate  and 

discuss  features  of  other  management  schemes 
and  what  problems  plagued  them.  Continue  by 
describing  how  the  Performance  Management 
Framework  can  overcome  these  problems. 

Lesson  Plan:  Introduction  5  min 

Ice  Breaker/  Stage  Setter 

-  Management  Techniques  10-15  min 

Components  of  a 

Successful  Management  System  5-10  min 

Roadblocks  to 

Successful  Management  Systems  5  min 

Course  Overview 

Framework  Objectives  2  min 

Course  Objectives  2  min 

Engineering  Environment 
Distinotives  5  min 

Role  of  Management 
Employee  Participation 
Training 

Target  of  Improvement  5  min 

Process  Framework 
Framework  Implementation 
Measurement  5  min 

Review  Framework  Objectives  1  min 
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Lesson  02  Potential  Benefits  from  Performance 

Improvement 

Time:  1  hour 

Objectives:  Detail  what  benefits  can  be  gained. 

Demonstrate  why  improvement  action  is  needed. 

Understand  benefits  experienced  in  another 
workplace  environment.  Discuss  differences 
in  the  civil  engineering  environment  that  may 
require  adaptation  or  changes  of  approach 
from  that  used  successfully  in  other 
situations . 

Technique:  Small  team  case  analysis  -  Critical 

evaluation  and  discussion  of  cases  by  small 
teams  with  presentation  of  conclusions  to 
complete  group. 

Lesson  Plan:  Introduction  2  min 

Case  Evaluations  (3  cases)  23  min 

Reports  (5-8  min  each)  20  min 

Application  (Time  permitting)  5  min 

-  Needs  for  improvement  in 
this  organization 
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Lesson*  03 
Time : 

ObJ  eetives : 

Technique : 


Lesson  Plan: 


Importance  of  Commitment  by  Top  Management 
1  hour 

Demonstrate  that  management  commitment  is 
crucial . 

Identify  ways  that  management  commitment  can 
be  increased. 

Paired  individuals  case  analysis  -  Study 
cases  and  determine  the  nature  and  extent  of 
management  commitment.  Suggest  how  and  why 
top  management  could  be  convinced  to  support 
the  Performance  Management  Framework  more 
fully. 

Introduction  5  min 

Case  Evaluations  (5  cases)  20  min 

Reports  (3-5  min  each)  20  min 

Application  5  min 

-  Commitment  of  top  management 
for  this  organization 
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Lesson  04 
Time : 

Ob j  ecti ves : 


Technique : 

Lesson  Plan: 


Management  Support  and  Cultural  Change 
1  hour 

Identify  the  support  roles  of  management  in 
implementing  and  maintaining  a  Performance 
Management  Framework. 

Define  the  concept  of  "Cultural  Change"  and 
communicate  the  essential  role  of  top 
management  in  creating  the  change  in  culture 
from  the  top  down. 

Structured  group  discussion  -  Lead  the  group 
through  an  outlined  format  presenting  the 
important  features  of  management  support  and 
cultural  change,  allowing  the  group  to 
contribute  insights  and  ideas. 


Review  and  Introduction 

3 

min 

Management  Support  Roles 

Provide  Example 

5 

min 

Provide  Resources 

3 

min 

Coordinate  Organization-Wide 
Efforts 

5 

min 

Reward  Results 

2 

min 

Cultural  Change 

Your  Organization’s  Culture 
-  Man  from  Mars 
Perspective 

20  min 

Change 

3 

min 

Formal  Channels 

3 

min 

Informal  Channels 

4 

min 

Conclusion  and  Summary 

2 

min 
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Lesson  05 
Time : 

Ob j ecti ves : 


Technique : 


Lesson  Plan: 


Teamwork  and  Participation  Concepts 
1  hour 

Establish  the  ability  of  a  team  to  do  more 
than  the  individual  can  in  the  subjective 
area  of  performance  improvement  planning. 

Identify  the  types  of  groups:  work  area 
teams  or  multi-functional  cross  disciplinary 
teams.  Discuss  the  effect  of  group  size  on 
group  dynamics. 

Present  common  problems  plaguing  groups. 
Teach  team  members  to  expect  them,  identify 
them,  and  correct  them. 

Croup  brainstorming  -  Allow  the  group  to 
generate  and  discuss  ideas  about  group 
effectiveness  and  difficulties.  Provide 
guidance  as  needed  to  direct  group  in 
covering  the  principal  concepts  listed. 

Introduction  3  min 

Group  Decision  Making 

Compared  to  Individual  12  min 

Group  Structure 


Composition 

8  min 

Size 

7  min 

Group  Roles 

12  min 

Conf 1 ict 

8  min 
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Lesson  06 


Team  Participation  Exercise 


Time : 

Ob j ect ives : 


Technique : 

Lesson  Plan: 


2  hours 

Allow  the  group  members  to  work  as  a  team  to 
solve  an  assigned  task.  Qive  the  individuals 
an  opportunity  to  observe  group  interaction 
techniques  and  problems. 

Discuss  the  dynamics  of  the  group  exercise. 
Identify  specific  roles  that  people  adopt  in 
a  group  environment  to  facilitate 
performance . 

Group  exercise  -  Lead  the  group  through  the 
Farmer  Exercise.  Discuss  the  results. 

Prompt  the  discussion  by  asking  about 
specific  roles  team  members  were  observed  to 


adopt . 

Farmer  Exercise  50  min 

Break  10  min 

Evaluation 

Review  Objectives  2  min 

Importance  of  Individual’s 

Contribution  5  min 

Understanding  and  Identifying 

Roles  28  min 

Faoi 1 i tators 

Introduction  5  min 

Response  to  Roles  10  min 
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Lesson  07 
Time : 

Ob j  ectives : 


Technique : 

Lesson  Plan: 


Training  Objectives  and  Pitfalls 
1  hour 

Establish  the  broad  training  goal  of  creating 
the  cultural  change  needed  for  a  successful 
program.  Define  the  more  specific  training 
goals  of  imparting  tools  for  use  in  the 
Performance  Management  Framework. 

Introduce  some  of  the  most  common  problems 
and  errors  encountered  in  training  programs. 

Group  brainstorming  -  Encourage  the  group  to 
suggest  objectives  and  problems  of  training. 
Provide  guidance  as  needed  to  direct  the 


session 

1 isted . 

through 

the  important 

concepts 

Introduction 

10 

min 

Cul tural 

Change 

Training 

15 

min 

Speci f ic 

Tools 

Training 

10 

min 

Problems 

15 

min 
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Lesson  08 
Time : 

Ob j  ecti ves : 


Technique : 

Lesson  Plan: 


Training  Techniques  and  Strategies 
1  hour 

Identify  which  individuals  in  the  Engineering 
Branch  should  receive  training.  Suggest  what 
factors  are  pertinent  to  choosing  these 
people . 

Determine  the  most  opportune  time  to  provide 
various  degrees  of  training.  List  important 
criteria  in  selecting  this  correct  time  in 
different  situations. 

List  the  essential  ingredients  to  be  included 
in  the  training  curriculum.  What  situational 
factors  might  require  revising  these 
ingredients? 

Small  team  presentations  -  Provide  each  of 
three  teams  with  a  list  of  questions  and 
objectives.  The  team  will  generate 
recommendations  and  present  them  to  the 
group . 


Introduction 

3  min 

Team  Project 

32  min 

Presentations 

15  min 
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Lesson  09 
Time  : 

Ob j  ecti ves : 


Technique : 

Lesson  Plan: 


Process  Improvement 
1  hour 

Present  the  concepts  associated  with  viewing 
organizational  activity  from  a  process 
perspective . 

Understand  the  five  principal  features  of 
process  improvement  plans. 

Define  cross-functional  processes.  Establish 
how  to  determine  input  and  output  which  flows 
between  these  producers  and  consumers. 

Discuss  ownership  of  processes.  Show  the 
importance  of  assigning  responsibility  for  a 
process  to  a  single  individual. 

Communicate  the  necessity  of  extending  the 
process  to  its  origin  and  conclusion.  The 
greatest  improvements  for  the  least  expense 
can  often  be  made  by  thorough  planning  at  the 
start  of  the  process. 

Structured  group  discussion  -  Follow  the 
format  provided,  encouraging  the  group  to 
expand  upon  and  define  terms  and  concepts 
essential  to  understanding  the  process 
perspective . 


Introduction 

3  min 

Process 

Perspective 

7  min 

Process 

Improvement  Plans 

15  min 

Cross-functional  Processes 

10  min 

Process 

Ownership 

5  min 

Process 

Extens i on 

10  min 
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Lesson  10  Process  Flow  Exercise 

Time:  1  hour 

Objectives:  To  allow  the  group  to  identify  a  process  in  a 

multi-functional  environment.  The  process 
interfaces  will  be  determined,  along  with  the 
input  and  output  passed  between  these 
functions . 

Technique:  Group  brainstorming  -  Suggest  a  situation  the 

members  of  the  group  will  be  familiar  with, 
perhaps  getting  up  in  the  morning,  getting 
ready  and  getting  to  work.  Have  the  group 
flow  the  process.  Ask  the  group  to  extend 
the  process  to  its  extreme  beginning  and 
conclusion.  Discuss  how  actions  taken  at  the 
beginning  can  dramatically  affect  the 
outcome . 

Lesson  Plan:  Introduction  3  min 

Process  Excercise 

Introduction  2  min 

Process  Flow  Diagraming  15  min 

Cross- f unct i onal  aspects 

Function  Identification  6  min 

Input  Identification  6  min 

Output  Identification  6  min 

Ownership  2  min 

Extension  10  min 
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Lesson  1 1 
Ti  me  : 

ObJ  ectives : 

Technique : 


Lesson  Plan: 


Process  Flow-charting 
1  hour 


To  provide  further  experience  in  describing 
the  underlying  processes  comprising  the 
workplace . 


Panel  Discussion  -  Assign  one  of  four  case 
studies  to  each  individual.  After  allowing 
time  for  reading  and  considering  the  case, 
call  panels  of  individuals  who  had  the  same 
case  to  identify  the  central  process(es) 
described.  Emphasize  the  cross-functional 
nature  of  the  processes.  Describe  the  input 
and  output  at  each  interface. 


Introduction 
Read  Cases 
Panel  Discussion 
Conclusion 


5  min 
5  min 
36  mi  n 
4  min 
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Lesson  12  Definition  of  Performance  &  Customer 

Satisfaction 

Time :  1  hour 

Objectives:  List  and  define  the  seven  components  of 

performance.  Discuss  which  are  most 
appropriate  for  the  Engineering  Branch. 

Understand  the  important  role  of  customer  or 
user  satisfaction  in  improving  performance. 
Identify  who  customers  are  of  the  Engineering 
Branch . 

Technique:  Group  brainstorming  -  Present  the  seven 

components  of  performance.  Allow  the  group 
to  suggest  the  value  of  each  in  their 
environment.  Permit  the  group  to  generate  a 
list  of  customers  and  how  the  Engineering 
Branch  serves  these  customers. 

Lesson  Plan:  Introduction  5  min 


Performance  Component  Definition 


Ef  f ectiveness 

4 

mi  n 

Ef  f iciency 

4 

min 

Productivity 

4 

mi  n 

Qual i ty 

4 

mi  n 

Innovation 

4 

min 

Quality  of  Work  Life 

4 

min 

Budgetabi 1 i ty 

4 

mi  n 

Component  Importance 

2 

min 

Customer  Satisfaction 

Importance  of  Customers 

5 

min 

Identification  of  Customers  10  min 


108 


Lesson  13 
Time : 

ObJ  eoti ves : 

Technique : 

Lesson  Plan: 


Application  of  Performance  Components 
1  hour 

Apply  the  components  of  performance  to  the 
Engineering  Branch  environment.  Suggest 
elements  of  performance  which  could  be 
targeted  for  evaluation  and  improvement  under 
the  Performance  Management  Framework. 

Paired  individuals  analysis  and  presentation 
-  Direct  each  pair  to  critically  evaluate 
their  organization  in  regard  to  one  of  the 
components  of  performance.  Their  conclusions 
will  then  be  briefed  to  the  group. 


Introduction 

5 

min 

Paired  application  analysis 

25 

min 

Presentations 

20 

min 
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Lesson  14 


Implementation  Details 


Time  : 

Ob j  ectives : 


Technique : 

Lesson  Plan: 


1  hour 

Outline  the  steps  involved  in  a  top-down 
implementation  of  a  Performance  Management 
Framework . 

Illustrate  the  technique  of  using  a  pilot 
project  to  initiate  the  program. 

Describe  the  importance  of  the  facilitator  to 
the  operation  of  the  team  function. 

Paired  individuals'  reports  -  Assign  one  of 
the  objectives  to  each  pair.  Provide 
guidance  from  which  they  can  prepare 
recommendations.  Have  each  pair  present 
their  findings  to  the  entire  group. 

Introduction  3  min 

Paired  individual  analysis  25  min 

Presentations  20  min 


Conclusion 


2  min 


Lesson  15 
Time : 

Obj  ectives : 


Technique : 

Materials : 

Lesson  Plan: 


Implementation  Exercise 
3  hours 

Provide  a  situation  in  which  members  can 
experience  implementation  of  the  Performance 
Management  Framework. 

Discuss  the  lessons  learned  and  how  they 
might  be  applied  to  the  program 
implementation  in  the  Engineering  Branch 
Environment . 

Direct  the  group  in  participation  in  the 
Project  Workshop.  Lead  the  group  in 
discussing  their  experience. 

Several  dozen  8-1/2  x  11  sheets  card  stock  or 
other  heavy  construction  paper  (buff  color  is 
fine),  masking  tape,  unsharpened  pencils, 
typing  paper,  scissors,  12  inch  scales, 
drafting  triangles. 

Introduction  5  min 

Exercise 


Situation 

30 

min 

Management  Initiation 

15 

min 

Break 

10 

min 

Team  Time  1 

25 

min 

Team  Time  2 

25 

min 

Break 

10 

min 

Team  Time  3 

25 

min 

Management  Implementation 

10 

min 

Debrief 

15 

min 

Lesson  16 
Time : 

Ob j  ecti ves : 


Technique : 

Lesson  Plan: 


Measurement  Exercise 
1  hour 


Understand  the  importance  of  measurement  in 
evaluating  and  improving  performance. 

List  reasons  for  measuring. 

Define  what  to  measure  as  the  products  which 
pass  between  steps  in  the  process ,  from 
internal  suppliers  to  internal  customers. 


Examine  a  method  for  analyzing  variance 
between  these  steps. 


Ham  and  Cheese  Sandwich  exercise  -  Guide  the 
group  in  the  exercise  and  discussion  of 
appl ications . 

Introduction  3  min 

Measurement 


Importance  3  min 

Reasons  for  6  min 

Variance 

External  3  min 

Internal  5  min 

Variance  Matrix  30  min 
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Lesson  17 


Time : 

Obj  ectives : 

Technique : 

Lesson  Plan: 


Cause  and  Effect  of  Measurement 
1  hour 

Understand  the  importance  of  measurement,  and 
the  components  of  an  effective  measurement 
system. 

Group  brainstorming  -  Have  the  group  develop 
a  cause  and  effect  diagram  for  an  effective 
measurement  system. 

Introduction  3  min 


Cause 

and 

Effect 

Technique 

17 

min 

Cause 

and 

Effect 

of  Measurement 

30 

min 
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Lesson  18  '  Objectives  Matrix 

Time:  1  hour 

Objectives:  Demonstrate  a  technique  to  summarize  non- 

related  measures  into  a  composite  score. 

Describe  the  distinction  between  product, 
process  and  surrogate  measures ,  and  the 
advantages  of  each. 

Technique:  Individual  Module  and  discussion  -  Have  each 

individual  complete  the  worksheet  module 

illustrating  the  objectives  matrix.  Allow 
the  members  to  discuss  its  use  and 
appl ication . 

Lesson  Plan:  Introduction  7  min 

Types  of  Measures  8  min 

Objectives  Matrix  Worksheet  25  min 

Discussion  10  min 
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Appendix  Student^s  Handbook 


This  appendix  contains  the  Student’s  Handbook  for  a 
three  day  training  course  in  performance  management.  It 
follows  the  Plan  of  Instruction  given  in  Appendix  A. 
Detailed  instructions  for  how  to  lead  a  class  through  the 
training  course  are  contained  in  the  Leader’s  Quide  in 
Appendix  C. 

An  introduction  and  schedule  of  instruction  are  given. 
Objectives  for  the  course  and  each  lesson  are  given.  Where 
appropriate,  outlines  for  the  lessons  are  given  with  room 
provided  for  note  taking.  The  Student's  Handbook  is 
intended  to  be  provided  to  each  student  at  the  beginning  of 
the  course.  The  student  would  then  be  allowed  to  take  the 
handbook  with  him  or  her  at  the  conclusion  of  the  course  as 
a  reference. 

In  preparing  this  handbook,  material  was  adapted  from 
other  sources  as  appropriate.  This  material  is  referenced 
below: 


Session 

2 

Case  Descriptions  (Hayes,  1990:19-24). 

¥ 

Session 

3 

Management  Cases  (Wertz,  1989; 
Warmington,  1988;  Simmons,  1990:74-76; 
Barra,  1989:46-50) 

m 

Session 

5 

Group  roles  (HQ  AFLC ,  1989d : 120- 121) . 
Conflict  responses  (Gray  and  Smeltzer, 
1989:412-413) . 

Session 

9 

Process  improvement  plans  (Jennings  and 
others ,  1989 : 12-89) . 

Session 

12 

Performance  components  (Sink  and  Tuttle 
1989: 171-186) . 

Se3s ion 

16 

Variance  matrix  (Jennings  and  others, 
1989:50-57) . 

Session 

17 

Cause  and  Effect  (Department,  1989a:51) 

Session 

18 

Objectives  matrix  (Riggs,  1987:648-660) 
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PERFORMANCE  MANAGEMENT  FRAMEWORK  HANDBOOK 


A  handbook  for  the  evaluation  and 
improvement  of  performance  in  the  Engineering  Branch  of 
Air  Force  Civil  Engineering  Squadrons 

FOREWARD 

This  handbook  is  designed  to  assist  Engineering  Branch 
managers,  engineers,  and  others  who  contribute  to  the 
preparation  and  execution  of  projects  for  real  property 
construction,  maintenance,  and  repair.  These  individuals 
are  challenged  with  providing  and  maintaining  quality 
facilities,  allowing  Air  Force  organizations  to  perform 
their  missions.  Evaluation,  management  and  improvement  of 
the  performance  of  Engineering  Branch  personnel  is  central 
to  continuing  to  fulfill  this  mandate. 

The  goal  of  this  handbook  and  the  training  which  accompanies 
it  is  to  acquaint  Engineering  Branch  members  with  a 
framework  for  managing  performance  and  to  provide  the  tools 


needed  to  use  this  framework. 


Performance  Management  Framework  Objectives: 

Foster  a  management  perspective  conducive  to 
performance  improvement 

Create  an  organizational  environment  promoting 
performance  improvement 

Continually  encourage  the  quest  to  discover  and 
implement  ways  to  improve  performance 


Course  Objectives: 

To  equip  civil  engineering  personnel  to  manage  a 
performance  management  framework 

To  create  momentum  for  performance  improvement 

To  promote  within  civil  engineering  personnel  a 
perspective  for  improvement 

To  provide  the  essential  tools  and  techniques  to 
establish  and  maintain  a  performance  management 
framework 


CLASS  SCHEDULE 


First  Day  Time  Page 

Morning 

Topic:  Introduction/Engineering 

Environment  Distinctives 

Session  1  Introduction/ 

Distinctives 

Session  2  Benefits  Available 

Topic:  Role  of  Management 

Session  3  Management  Commitment  1  hour  126 

Session  4  Management  Support  1  hour  134 

Lunch  1  hour 

Af  ternoon 

Topic:  Employee  Participation 

Session  5  Team  Participation  1  hour  136 

Session  6  Participation  Exercise  2  hours  140 


1  hour  120 

1  hour  121 


1 18 


Second  Day 

Time 

Page 

Morning 

Topic : 

Training 

Session 

7 

Training  Objectives 

1 

hour 

143 

Session 

8 

Training  Techniques 

1 

hour 

145 

Topic : 

Process 

:  Framework 

Session 

9 

Process  Improvement 

1 

hour 

146 

Session 

10 

Process  Flow  Exercise 

1 

hour 

149 

tr 

Af  ternoon 

Lunch 

1 

hour 

M 

Session 

1 1 

Process  Flow-charting 

1 

hour 

150 

Topic : 

Target 

of  Improvement 

Session 

12 

Performance  Definition 

1 

hour 

151 

Session 

13 

Performance  Components 

1 

hour 

157 

Third  Day 

Morning 

Topic : 

Framework  Implementation 

Session 

14 

Implementation  Details 

1 

hour 

158 

Session 

15 

Implementation  Exercise 

3 

hours 

160 

Lunch 

1 

hour 

A f  ternoon 

Topic : 

Measurement 

Session 

16 

Measurement  Exercise 

1 

hour 

162 

Session 

17 

Cause  and  Effect 

Discussion 

1 

hour 

168 

Session 

18 

Objectives  Matrix 

1 

hour 

170 

119 


Session  1  Introduction/Engineering  Environment 

Distinctives 


Objectives:  Distinguish  a  performance  management 

framework  from  other  management  fads  and 
gimmicks  which  have  come  and  gone. 

Establish  that  this  framework  i3  one  which 
civil  engineering  personnel  design  themselves 
to  fit  their  own  particular  situations. 


List  management  techniques  which  the  Air  Force  or  your 
organization  have  tried.  How  well  have  they  worked? 


What  are  some  essential  aspects  of  a  successful  management 
framework? 


What  are  some  aspects  of  management  techniques  which  have  not 
worked  or  have  hampered  the  effectiveness  of  the  technique? 
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Potential  Benefits  from  Performance 
I mprovement 

Detail  what  benefits  can  be  gained. 

Demonstrate  why  improvement  action  is  needed. 

Understand  benefits  experienced  in  other 
workplace  environments.  Discuss  differences 
in  the  civil  engineering  environment  that  may 
require  adaptation  or  changes  of  approach 
from  that  used  successfully  in  other 
s i tuations . 

The  three  case  descriptions  -  "Three  Views  of 
TQM,"  by  Glenn  E.  Hayes,  Quality,  Volume  29, 
April  1990,  pp  19  -  24. 

Questions  for  Case  Evaluations: 

What  need  did  this  organization  identify  which  they 
sought  to  improve? 


What  action  did  they  take  to  make  improvements?  What 
tools  were  used?  What  resources  did  they  tap? 


What  benefits  did  they  achieve?  What  improvements  were 
real ized? 


Session  2 


Ob j  ectives : 


Reference : 


How  did  this  environment  differ  from  the  Air  Force 
Engineering  Branch  environment  you  are  in?  In  what 
ways  was  it  similar? 


Would  the  approach  used  work  in  your  organization?  Why 
or  why  not?  What  would  need  to  be  done  differently  to 
make  improvements  to  some  area  of  need  in  your  branch? 
What  could  be  done  in  a  similar  fashion? 
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Case  1.  General  Dynamics  Space  Systems  Div. ,  San  Diego,  CA 

In  1985,  senior  managers  at  General  Dynamics  decided  to 
use  performance  management  principles  to  make  their  division 
prosper.  A  quality  team  was  created  to  examine  the 
organization.  Work  processes  were  found  which  were  stuck 
in  the  trap  of  always  doing  things  the  same  without 
questioning  whether  there  was  a  better  way.  An  atmosphere 
was  created  to  encourage  making  changes  without  workers 
fearing  reprisals. 

The  success  of  the  performance  improvement  effort  was 
attributed  to  a  large  degree  to  the  support  and  direction 
provided  by  the  top  management.  This  group  showed 
leadership,  not  just  management,  in  making  sure  the  employee 
improvement  teams  had  all  they  needed  to  tackle  nagging 
problems  and  bring  misunderstood  work  processes  to  light. 

Decisions  were  made  about  training,  including  who, 
what,  when  and  how.  Senior  management  sought  training  in 
how  to  provide  a  more  favorable  environment  in  which  the 
workers  could  proceed  with  the  performance  improvement 
efforts.  Management’s  commitment  was  shown  further  by 
making  sure  everyone  else  also  received  the  training  they 
needed  in  order  to  begin.  Then  management  3tood  back  and 
let  teams  of  workers  dive  into  evaluating  and  improving 
their  work  processes  using  performance  management 
techniques . 

As  a  result,  the  team’s  departments  are  now  working 
together  in  better  harmony.  The  material  procurement, 
manufacturing,  production  and  quality  departments  are 
functioning  more  as  a  single  unit.  They  communicate  more 
frequently  and  effectively.  Costs  have  been  reduced  by  as 
much  as  30X  as  well.  Improvements  were  made  not  only  on  the 
factory  floor,  but  in  the  administrative  processes.  Prior 
to  using  performance  management,  their  work  to  produce 
launch  vehicles  and  to  provide  launch  services  was 
understood  only  from  an  engineering  viewpoint.  However,  it 
took  the  new  techniques  and  mindset  of  performance 
management  to  understand  the  work  from  a  process  viewpoint. 
This  allowed  the  process  to  be  charted  and  areas  needing 
improvement  to  be  exposed. 

The  improvements  with  the  largest  potential  impacts 
were  those  focused  not  on  production  floor  but  on  the 
administrative  functions.  Paperwork  processes  were  found  to 
have  the  longest  cycle  times.  Shortening  these  turnaround 
times  proved  more  important  than  the  more  direct  production 
work  processes. 
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Case  2.  TRW  Space  &  Defense  Sector,  Redondo  Beach,  CA 

In  mid-1989,  TRW  set  goals  for  the  sector  into  the  new 
decade  and  chose  performance  management  as  a  way  to  help 
reach  these  goals.  One  reason  given  was  to  meet  their 
customers’  expectations.  A  growing  commitment  to 
performance  management  principles  exists  in  government,  with 
whom  much  of  their  work  is  done.  They  did  not  want  to  fall 
behind  their  competitors,  many  of  whom  have  begun  such 
efforts.  A  second  reason  given  was  the  bureaucratic  nature 
of  their  business.  Work  processes  needed  to  be  improved  to 
reduce  the  time  and  cost  associated  with  this  business 
bureaucracy.  Wasted  steps  were  to  be  cut  out  of  the 
processes,  promising  savings  of  up  to  30/C.  Worker 
motivation  would  be  increased  as  individual  responsibility 
was  restored  and  waste  eliminated. 

Top  management  supported  performance  management  by 
promoting  the  mindset  needed  to  look  for  improvement,  not 
just  to  get  by.  Involvement  by  every  manager  was  expected. 
Each  was  to  be  receptive  to  improvement  ideas  and  encourage 
the  efforts  of  their  people.  Further  support  was  provided 
in  the  form  of  resources  needed  to  implement  improvement 
teams  and  their  ideas. 

Performance  management  was  formally  begun  when  all 
senior  managers  attended  training  in  how  to  lead  the  effort. 
Process  action  teams  were  formed  and  training  provided  for 
their  members.  These  teams  were  directed  to  evaluate  and 
propose  improvements  in  the  work  processes  of  the 
organization.  A  goal  of  30/C  cost  reduction  over  several 
years  was  set.  Each  department  was  made  responsible  for  a 
portion  of  that  goal  each  year. 

Accomplishments  were  made  in  both  production  oriented 
and  idea  oriented  processes.  Six  processes  that  were 
initially  attacked  were:  1) travel  expense  reports;  2)  Time 
card  processing;  3)  New  hires  and  transfers;  4)  Facilities 
requests;  S)  Performance  measurement  systems;  and 
6)  Administrative  support  requests.  Each  was  marked  by 
bureaucracy,  and  improvements  were  made  by  streamlining  the 
processes  and  removing  areas  prone  to  mistakes. 

The  sector  made  sure  that  improvements  in  schedule, 
performance  or  cost  were  not  made  at  the  expense  of  quality. 
A  focus  was  maintained  of  improving  the  work  processes,  and 
not  just  trying  to  dress  up  the  products  that  result.  A 
strong  customer  involvement  was  developed,  making  sure  the 
work  was  in  harmony  with  customer  expectations. 
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Case  3.  M/A-COM  Government  Systems  Div.,  San  Diego,  CA 

M/A-COM  expanded  its  performance  management  efforts 
in  March,  1989  as  a  result  of  observing  improvements 
previously  achieved  in  a  limited  use  of  the  principles  in 
its  production  area.  Products  were  completed  with  less 
labor,  fewer  defects,  and  lower  inspection  costs.  An 
additional  motivation,  as  a  government  supplier,  was  the 
government’s  requirement  that  all  its  suppliers  use  such 
performance  management  principles. 

Decision  making  authority  was  spread  to  lower  levels  in 
the  organization.  Workers  were  recognized  for  their 
contributions  in  performance  management,  setting  the  example 
that  management  was  behind  such  efforts.  Top  management’s 
active  involvement  in  the  improvement  efforts  was  further 
demonstrated  by  including  the  topic  and  efforts  in  the 
executive  council  agenda  for  their  monthly  meetings. 
Initially,  critical  work  processes  to  be  evaluated  and 
improved  were  defined  by  this  council.  Since  then,  other 
processes  have  been  added  as  a  result  of  suggestions  from 
the  employees. 

Top  management  has  been  involved  in  activities  related 
to  performance  management  to  show  their  support.  Funds  have 
been  budgeted  for  training,  which  has  been  provided  to  all 
performance  improvement  teams.  The  efforts  were  given 
publicity  through  the  organization’s  publ ications ,  where 
teams  were  credited  for  their  accomplishments. 

A  consultant  was  hired  to  train  the  employees  in  the 
tools,  mindset,  and  concepts  of  performance  management.  The 
plan  was  publicized  to  all  personnel.  Teams  have  already 
improved  the  data-handl ing  process,  and  the  employee 
suggestion  policy  is  being  evaluated.  Perhaps  one  of  the 
most  significant  accomplishments  has  been  getting  employees 
to  work  together  in  teams.  Barriers  are  breaking  down  and 
communication  improving  as  they  understand  their  work 
processes  and  why  others  involved  in  them  do  the  things  they 
do.  Not  all  improvements  were  large  ones,,  but  as  long  as 
they  are  aimed  in  the  right  direction  they  have  been 
welcomed. 

A  critical  element  in  the  team  problem  solving  was  to 
first  properly  identify  the  process.  This  was  done  using  the 
organization’s  mission  statement  and  process  flow-charting. 
The  urge  to  jump  the  gun  needed  to  be  controlled.  Only 
after  adequate  training  and  thorough  process  definition 
could  successful  efforts  be  pursued.  Using  the  employees 
in  an  interactive,  participative  team  setting  has  unlocked 
much  potential  for  success. 
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What  areas  in  need  of  improvement  can  you  identify  in  your 
organization? 
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Session  3  Importance  of  Commitment  by  Top  Management 


Objectives:  Demonstrate  that  management  commitment  is 

crucial . 

Identify  ways  that  management  commitment  can 
be  increased. 

References:  Case  Descriptions  -  Design  and  Implementation 

of  Total  Quality  Management  i^n  a  Civil 

i,ng  Sguadron ,  by  1st  Lt  Robert  M. 
Wertz ,  1989 . 

-  Lessons  Learned  from  the  Implementation  of 
Total  Quality  Management  at  the  Naval 
Aviation  Depots  North  Island^  C A ,  by  Jeffery 
Allen  Warmington,  1988. 

-  'FPL  Wins  the  Deming  Prize,’  by  John 
Simmons,  The  Journal  for  Quality  and 
Participation ,  March  1990,  pp  74  -  76. 

-  'Motorola’s  Approach  to  Quality,’  by  Ralph 
Barra ,  The  Journal  for  Quality  and 
Participation,  Volume  12,  Number  1,  March 
1989,  pp  46  -  50. 

-  A  Performance  Management  Framework  for 
Civil  Engineering,  by  Robert  M.  Gill,  1990. 


Who  is  the  top  management  for  your  organization? 


How  critical  to  your  work  is  their  support? 


How  does  your  management  show  what  they  support? 
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Questions  for  Case  Evaluations: 


How  committed  was  top  management  in  this  case  to  the 
task  the  organization  was  working  on?  How  did 
management  show  their  level  of  commitment? 


What  would  you  suppose  management  was  committed  to? 


How  could  management’s  commitment  have  been  increased? 
Suggest  some  specific  ways. 


Who  could  have  the  possible 
increase  their  commitment? 
more  fully  support  the  task 


impact  on  top  management  to 
Who  could  convince  them  to 
of  the  organization? 
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Case  1.  2750  CES/DEM,  Wr  ight-Patterson  AFB ,  OH 


Early  in  fiscal  year  1989,  the  Operations  and 
Maintenance  Branch  of  the  2750  Civil  Engineering  Squadron 
began  a  formal  performance  management  effort.  The  squadron 
is  a  part  of  Air  Force  Logistics  Command,  which  at  the 
direction  of  its  Commander,  was  implementing  Total  Quality 
Management  principles  command  wide.  The  squadron  had 
established  a  quality  committee,  comprised  of  many  of  the 
top  officers  and  supervisors.  This  council  was  chaired  by 
the  Deputy  Base  Civil  Engineer,  and  oversaw  the  quality 
efforts  of  the  squadron.  The  Industrial  Engineering  Branch 
of  the  squadron  had  several  enthusiastic  and  capable  members 
who  were  eager  to  begin  performance  management  efforts 
squadron  wide. 

The  quality  committee’s  job  was  to  promote  improvement 
within  the  organization  and  to  determine  when  to  form 
process  action  teams  and  which  processes  should  be  assigned 
for  analysis.  Committee  members  attended  team  meetings  on 
occasion  to  show  support  and  observe  the  activity.  The 
committee  designated  a  large  conference  room  for  team 
meetings.  They  further  allowed  team  members  time  to  meet, 
usually  an  hour  per  week.  When  a  team  arrived  at  proposed 
solutions  for  improvements,  the  committee  was  briefed  by  the 
team.  The  committee  then  took  action  on  these 
recommendations . 

As  the  effort  progressed,  and  teams  which  were  formed 
had  been  meeting  for  some  time,  the  quality  committee  was 
found  to  provide  inadequate  direction  to  the  process 
improvement  efforts.  One  employee  expressed  a  feeling  that 
were  it  not  for  the  AFLC  commander’s  personally  showing 
Interest  and  involvement,  the  input  from  the  Civil 
Engineering  Squadron’s  top  management  would  be  almost 
nothing.  It  was  only  because  the  Industrial  Engineering 
Branch  advisors’  efforts,  covering  up  for  lack  of  top 
squadron  support,  that  any  successful  results  were  obtained. 
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Case  2.  Naval  Aviation  Depot,  North  Island,  CA. 

In  1984,  management  at  the  Naval  Aviation  Depot  decided 
to  begin  using  performance  management  principles  to  improve 
their  work  processes.  Initial  training  was  slated  for  only 
the  upper  and  middle  management  personnel.  No  training  was 
provided  for  the  workers  and  their  immediate  supervisors. 

The  only  training  these  employees  received  was  that  which  was 
passed  down  by  upper  management. 

The  top  management  at  the  depot  delegated  the  control 
of  the  performance  management  effort  to  a  staff  group,  while 
the  attention  of  top  management  was  then  diverted  elsewhere. 
This  sent  the  signal  to  the  rest  of  the  organization  that 
the  new  improvement  plans  were  not  a  top  priority,  and  that 
top  management  was  not  supporting  the  performance  management 
e  f  f  or t . 

Process  action  teams  were  formed  throughout  the 
organization.  In  fact,  the  explosion  in  the  number  of  new 
teams  being  formed  grew  too  fast  for  upper  management  to 
support  them  effectively.  Not  enough  planning  or  control 
had  been  established  to  direct  the  performance  improvement 
momentum  once  it  had  been  established.  In  spite  of  the 
organizational  time  and  resources  going  into  performance 
management,  pressure  was  still  being  applied  from  users  of 
the  depot  for  all  the  existing  quotas  to  be  met.  Top 
management  was  not  supporting  the  teams  by  coordinating  new 
schedules  of  delivery  with  these  users,  allowing  the  teams 
time  to  get  established. 

The  employees’  performance  appraisals  were  not  changed 
to  reflect  the  new  responsibilities  of  the  process 
improvement  teams.  Soon  the  effectiveness  of  the  effort 
began  to  decrease.  Without  support  from  top  management  for 
a  change  in  emphasis  for  the  organization,  interest  in 
working  for  improvements  trailed  off. 
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Case  3.  Florida  Power  and  Light 

In  1981,  Florida  Power  and  Light  began  using  quality 
teams  to  improve  their  delivery  of  electrical  power  to  its 
customers.  They  achieved  impressive  results,  being  honored 
international iy  by  winning  the  Deming  Prize,  awarded  to  the 
organization  using  performance  management  techniques  best. 
The  transformation  at  Florida  Power  and  Light  was  led  all 
the  way  by  its  top  management.  The  senior  management  team, 
led  by  the  company’s  chairman,  showed  their  commitment  to 
new  ways  of  managing  by  their  behavior  and  their  decisions. 

The  senior  management  team  journeyed  to  Tokyo,  Japan, 
to  discuss  performance  improvement  issues  with  a  team  of 
counselors  there.  Most  any  week  of  the  year  one  of  these 
Japanese  counselors  was  likely  to  be  found  in  Florida 
working  with  some  unit  of  the  organization.  They  worked  to 
install  a  management  system  similar  to  what  utility 
companies  in  Japan  had  found  successful.  The  system 
emphasized  getting  ideas  for  improvements  from  the  workers 
to  management  and  the  other  way  around,  too. 

Each  senior  manager  is  assigned  objectives  for 
improvement  across  several  departments,  to  substitute 
cooperation  for  conflict.  These  managers  made  customer 
satisfaction  their  goal,  and  pushed  this  goal  down  through 
the  levels  of  management  in  the  organization. 

Management  decided  to  enter  the  competition  for  the 
Deming  Prize  as  an  incentive  for  their  company  to  work  for 
even  greater  improvements.  The  organization  already  had 
several  years  experience  in  performance  management  by  this 
time.  Their  efforts  accelerated  as  each  member,  already 
familiar  with  the  performance  management  principles, 
multiplied  their  involvement  toward  this  end.  As  a  result, 
they  accomplished  in  one  year  what  it  might  have  taken  three 
or  four  to  accomplish  otherwise. 

Senior  management  was  committed  throughout,  going  to 
classes,  and  refusing  to  delegate  the  reviews  of  progress, 
but  rather  performed  these  themselves.  They  had  the 
patience  not  the  abandon  the  plan  when  quick  results  did  not 
materialize.  They  remained  dedicated  to  providing  the 
tremendous  investment  needed  to  train  all  employees. 
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Case  4.  Motorola,  Inc. 


Motorola  began  their  performance  management  effort  in 
1980.  They  have  emphasized  a  top-down  plan  and  a  bottom-up 
dedication  of  employees  to  that  plan.  Their  first  step  was 
to  form  a  top-management  council  of  quality  executives. 

This  council  considered  how  to  employ  their  most  valuable 
resource  for  performance  improvement,  their  people. 

An  ambitious  training  program  was  instituted  to  speed 
up  the  improvement  process.  This  included  not  only 
analytical  tools  but  quality  awareness  training  as  well.  In 
addition,  hiring  criteria  was  changed  to  bring  in  more 
quality  conscious  people. 

Motorola’s  twice-a-quarter  policy  and  operating 
committee  meetings  focusing  on  performance  management  were 
chaired  by  no  less  than  the  Chief  Executive  Officer  of  the 
organization.  Progress  in  improvement  initiatives  was 
reviewed,  and  reports  made  by  each  manager  of  a  major  unit 
regarding  the  quality  efforts  of  that  unit.  Successes  as 
well  as  failures  are  briefed.  In  addition,  each  business 
unit  has  its  own  individual  performance  meeting,  again 
twice-a  quarter,  and  again  chaired  by  the  CEO,  to  look  in 
more  detail  at  that  unit’s  efforts. 

Performance  management  planning  is  not  limited  to  the 
top  officers  of  the  company.  A  program  for  participative 
management  encourages  every  employee  to  contribute 
suggestions  to  teams,  which  then  make  recommendations  for 
improvements.  This  fosters  management  support  and 
communication.  Managers  share  information,  problems  and 
opportunities  with  the  employees,  and  ask  for  their  ideas. 

Management  arranged  for  financial  savings  from  the 
improvement  efforts  to  be  shared  with  the  employees  as  part 
of  an  incentive  plan.  In  addition,  CEO  awards  are  are 
issued  to  recognize  large  achievements. 
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Case  5. 


2750  CES/DEE,  Wr ight-Patterson  AFB ,  OH 


At  the  beginning  of  the  1990  fiscal  year,  the  chief  of 
the  Engineering  Branch  of  the  2750  Civil  Engineering 
Squadron  decided  to  use  performance  management  in  the 
organization  to  solve  a  persistent  problem  with  the  design 
review  process.  Performance  management  principles  were 
chosen  because  of  growing  pressure  from  the  Air  Force 
Logistics  Command  to  see  organizations  turn  to  these 
techniques,  and  to  process  action  teams,  to  improve  work 
processes.  Support  was  provided  by  an  individual  in  the 
Industrial  Engineering  Branch.  Initial  training  was 
provided  to  all  team  members  by  this  individual. 

The  Engineering  Branch  chief  personally  picked  a  team 
of  individuals  from  the  branch  to  participate  in  the  team 
improvement  effort.  The  chief  decided  to  personally  be  a 
member  of  the  team  as  well.  The  chief’s  office  was  used  for 
meetings,  and  the  team  members  were  directed  to  take  time 
out  of  their  schedules  to  attend  and  contribute.  The  chief 
demonstrated  by  personal  attendance,  interest  and 
involvement  in  the  discussions  of  the  process,  that 
management  commitment  was  present.  Willingness  to 
immediately  implement  the  recommendations  of  the  team 
further  showed  this  commitment. 

The  chief’s  influence  was  used  as  needed  to  bring  input 
or  attendance  from  representati ves  of  other  organizations  to 
meetings.  This  allowed  the  team  to  get  at  the  root  of  the 
problems  they  encountered.  Memo3  were  drafted  and  meetings 
set  up  to  communicate  the  results  of  the  team’s 
investigation.  The  team  members  seemed  to  put  even  more 
energy  into  evaluating  and  improving  the  design  review 
process  after  seeing  their  chief  take  such  an  enthusiastic 
interest  in  the  whole  business. 

When  the  team  had  reached  their  conclusions,  an 
opportunity  was  arranged  to  brief  the  squadron  quality 
committee  on  the  results.  This  committee  was  made  up  of 
many  of  the  senior  officers  and  supervisors  from  the 
squadron.  However,  at  the  appointed  time,  most  all  the 
members  of  the  committee  were  tied  up  elsewhere,  and  the 
presentation  of  results  to  management  was  of  little  impact. 
Nevertheless,  the  Engineering  Branch  chief  continued  to  do 
all  possible  to  see  that  the  team’s  efforts  were 
implemented . 
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How  committed  is  your  top  management  to  a  performance 
management  framework? 


How  will  this  level  of  support  impact  your  ability  to  use 
a  performance  management  framework? 


How  can  commitment  by  management  be  increased,  if  needed,  or 
maintained  if  already  adequate?  Who  can  best  convince  your 
top  management  in  this  regard? 
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Session  4  Management  Support  and  Cultural  Change 

Objectives:  Identify  the  support  roles  of  management  in 

implementing  and  maintaining  a  performance 
management  framework. 

Define  the  concept  of  "Cultural  Change’  and 
communicate  the  essential  role  of  top 
management  in  creating  the  change  in  culture 
from  the  top  down. 

Management  Support  Roles 
Provide  Example 

Show  interest 
Get  involved 

Encourage  individual  applications 
Apply  the  framework  at  the  top 
Make  visible  changes 
Provide  Resources 
Time 
Space 
Funds 

Computer  Resources 
Support  from  consul tants/DEI 
Coordinate  Organization-Wide  Efforts 
Mul ti -d iscipl ine  applications 
Planning 

Assure  efforts  are  being  made 
Select  processes  for  improvement 
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Reward  Results 


Recognition 

Awards 

Access  to  resources 
Cultural  Change 
Cul ture 

Your  Organization’s  Culture  -  Man  from  Mars  Perspective 


Unneeded  cultural  attributes 


Change 

Formal  Channels 

Make  new  policy 
Counter  old  culture 
Informal  Channels 

Identify  informal  group  leaders 
Solicit  their  assistance 
Conclusion  and  Summary 
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Session  5 
Obj  ectives : 


Re  f  erences  : 


Teamwork  and  Participation  Concepts 

Establish  the  ability  of  a  team  to  do  more 
than  the  individual  can  in  x,he  subjective 
area  of  performance  improvement  planning. 

Identify  the  types  of  groups:  work  area 
teams  or  multi-functional  cross  disciplinary 
teams.  Discuss  the  effect  of  group  size  on 
group  dynamics. 

Present  common  problems  plaguing  groups . 
Teach  team  members  to  expect  them,  identify 
them,  and  correct  them. 

Group  role  definitions  -  Student^s  Gui^de^ 
AFLC  Facilitator  Training,  1st  Edition, 
October  1989,  pp  120-121. 

Conflict  response  strategies  -  Management j. 
The  Competitive  Edge,  by  Edmund  R.  Gray  and 
Larry  R.  Smeltzer,  1989,  pp  412-413. 


Individual  Decision  Making  Group  Decision  Making 


Advantages 


Advantages 


Disadvantages 


Disadvantages 


Situational  Factors 


Situational  Factors 


* 


Application  Opportunities  Application  Opportunities 
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When  might  a  functional -area  group  be  appropriate? 


When  might  a  cross- functional  group  be  preferred? 


How  does  group  size  affect  group  effectiveness? 


Common  Group  Roles: 

Task  Related: 

Initiator  -  Proposes  tasks  or  goals;  defines  a  group 

problem;  suggests  a  procedure  or  idea  for  solving 
a  problem 

Seeker  -  Requests  facts;  seeks  relevant  information 
about  group  concern;  asks  for  expressions  of 
feelings;  requests  a  statement  or  estimate; 
solicits  expressions  of  value;  seeks  suggestions 
and  ideas 

Giver  -  Offers  facts;  provides  relevant  information 
about  group  concern;  states  a  belief  about  a 
matter  before  the  group,  giving  suggestions  and 
ideas 

Summarizer  -  Interprets  ideas  or  suggestions;  clears  up 
confusion;  defines  terms;  indicates  alternatives 
and  issues  before  the  group;  pulls  together 
related  ideas;  restates  suggestions  after  the 
group  has  discussed  them,  offering  a  decision  or 
conclusion  for  the  group  to  accept  or  reject;  asks 
to  see  if  group  is  nearing  a  decision 
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Maintenance  Related: 


Harmonizer  -  Attempts  to  reconcile  disagreements; 
reduces  tension;  gets  people  to  explore 
di f  f erences 

Qate  Keeper  -  Helps  keep  communication  channels  open; 

facilitates  the  participation  of  others;  suggests 
procedures  that  permit  sharing  remarks 

Encourager  -  Acts  friendly,  warm,  and  responsive  to 
others;  indicates  acceptance  of  others’ 
contributions 

Compromiser  -  Offers  a  compromise;  admits  error; 

modifies  own  ideas  in  interest  of  group  cohesion 
or  growth 

Standard  Tester  -  Tests  whether  group  is  satisfied  with 
its  procedures;  points  out  explicit  or  implicit 
norms  which  have  been  set  to  make  them  available 
for  testing 

Self  Related: 

Aggressor  -  Deflates  the  status  of  others;  disapproves 
values,  attacks  the  group  problem;  jokes 
aggressively;  seeks  recognition 

Blocker  -  Tends  to  be  negativistic  and  stubbornly 

resistant;  disagrees  and  opposes  beyond  reason; 
attempts  to  bring  back  issue  after  group  has 
rejected  it;  refuses  to  or  ceases  to  participate 

Dominator  -  Tries  to  assert  authority  in  manipulating 
the  group  or  certain  members;  flatters;  asserts  a 
superior  right  to  attention 

Playboy  -  Makes  a  display  of  his/her  lack  of  ability 
and  involvement  in  the  group  process;  cynical  or 
nonchalant;  engages  in  horseplay 
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What  are  the  merits  and/or  problems  with  each  of  the 
following  ways  of  dealing  with  conflict? 

Separate  the  conflicting  parties 


Impose  rules  or  regulations  to  reduce  the  conflict 


Bring  the  parties  together  to  confront  the  issues  and 
work  out  solutions 
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Session  6  Team  Participation  Exercise 

Objectives:  Allow  the  group  members  to  work  as  a  team. 

Give  the  individuals  an  opportunity  to 
observe  group  interaction  techniques  and 
problems . 

Discuss  the  dynamics  of  the  group  exercise. 

Identify  specific  roles  that  people  adopt  in 
a  group  environment  to  facilitate 
performance . 

Discover  the  role  of  a  facilitator  for  the 
group . 


How  was  this  exercise  similar  to  team  decision  making  you 
might  encounter  on  the  Job? 


Role  Who  ?  How? 

Initiator 


Seeker 


Giver 


Summarizer 
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How? 


Role  Who? 

Harmonlzer 

Gate  Keeper 

Encourager 

Compromiser 

Standard  Tester 

Aggressor 

Blocker 

Dominator 

Playboy 

What  is  a  facilitator? 


What  actions  might  a  facilitator  take  in  a  group  decision 
making  session? 
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How  might  a  facilitator  respond  to  each  of  these  roles? 
Initiator 

Seeker 

Giver 

Summarizer 

Harmonizer 

Gate  Keeper 

Encourager 

Compromiser 

Standard  Tester 

Aggressor 

Blocker 

Dominator 

Playboy 
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Session  7 


Training  Objectives  and  Pitfalls 


Objectives:  Establish  the  broad  training  goal  of  creating 

the  cultural  change  needed  for  a  successful 
program. 

Define  the  more  specific  training  goals  of 
imparting  tools  for  use  in  a  performance 
management  framework. 

Introduce  some  of  the  most  common  problems 
and  errors  encountered  in  training  programs. 


What  is  cultural  change? 


How  can  training  be  used  to  begin  this  change? 


Where  should  this  training  be  accomplished? 


How  might  training  for  use  of  specific  performance 
management  framework  tools  differ  from  the  cultural  change 
training? 


Who  would  you  recommend  to  lead  this  training?  Where  might 
this  training  be  accomplished? 
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What  problems  might  arise  from  training  everyone  right  away? 


What  problems  do  you  foresee  from  training  people  too 
extensively,  or  in  too  much  detail  for  their  needs? 


What  problems  would  be  associated  with  having  no  training? 
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Session  8 


Training  Techniques  and  Strategies 


Ob j ectives : 


* 


Identify  which  individuals  in  the  Engineering 
Branch  should  receive  training.  Suggest  what 
factors  are  pertinent  to  choosing  these 
people . 

Determine  the  most  opportune  time  to  provide 
various  degrees  of  training.  List  important 
criteria  in  selecting  this  correct  time  in 
different  situations. 

List  the  essential  ingredients  to  be  included 
in  the  training  curriculum.  What  situational 
factors  might  require  revising  these 
ingredients? 
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Session  9 


Process  Improvement 

Objectives:  Present  the  concepts  associated  with  viewing 

organizational  activity  from  a  process 
perspective . 

Understand  the  five  principal  features  of 
process  improvement  plans. 

Define  cross- functional  processes.  Establish 
how  to  determine  the  input  and  output  which 
flow  between  these  producers  and  consumers. 

Discuss  ownership  of  processes.  Show  the 
importance  of  assigning  responsibility  for  a 
process  to  a  single  individual. 

Communicate  the  necessity  of  extending  the 
process  to  its  origin  and  conclusion.  The 
greatest  improvements  for  the  least  expense 
can  often  be  made  by  thorough  planning  at  the 
start  of  the  process. 

Reference:  Process  improvement  plans  -  Total  Quality 

Assurance  Through  Process  Management ,  by 
Kenneth  R.  Jennings  and  others,  1989,  12-89. 


Process  Perspective 
Definition 


Repeatable 


Universal 


Relate  to  big  picture 
Building  blocks  of  performance 
Key  to  improvement 
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Process  Improvement  Plans 


Definition 


Suppliers  Activity  Steps 

Input - >  Transformation  -- 


Simpl i f ication 


Measurement 


Control 


Improvement 

Identify  Opportunities 


Cause  and  Effect  Planning 


Action 


Users 
->  Output 
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Cross-functional  Processes 

Functional  interdependence 

Input 

Output 

Coordination 

Process  Ownership 

Responsibility 

Authority 

Pride 

Process  Extension 
Downstream 

Upstream 


Di f  f lculties 
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Session  10 
ObJ  ectives : 


Process  Flow  Exercise 

To  allow  the  group  to  identify  a  process  in  a 
multi-functional  environment.  The  process 
interfaces  will  be  determined,  along  with  the 
input  and  output  passed  between  these 
functions . 
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Session  11 


Process  Flow-charting 


Objectives:  To  provide  further  experience  in  describing 

the  underlying  processes  comprising  the 
workplace . 

Summarize  the  case. 


Describe  the  processes  evident  in  the  case.  List  the  steps 
which  comprise  these  processes. 


Which  separate  functional  units  are  interfaced  with  in  each 
process  ? 


Who  is  the  process  owner?  Over  which  steps  in  the  process 
does  the  owner  have  personal  authority? 


What  are  elements  of  input  and  output  evident  in  these 
processes? 


4 


How  can  defining  the  process  in  work  situations  assist  in 
managing  and  improving  the  organization’s  performance? 
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Session  12 


Objectives: 


Re  f  erence : 


Definition  of  Performance  &  Customer 
Satisfaction 

List  and  define  the  seven  components  of 
performance.  Discuss  which  are  most 
appropriate  for  the  Engineering  Branch. 

Understand  the  important  role  of  customer  or 
user  satisfaction  in  improving  performance. 
Identify  who  customers  are  of  the  Engineering 
Branch . 

Planning  and  Measurement  in  Your  Organization 
of  the  Future,  by  D.  Scott  Sink  and  Thomas 
C.  Tuttle,  1989,  ppl71-186. 


Performance  Components 

Ef  f  ectiveness 

Operational  Definition  - 

The  ratio  of  actual  output  to  expected 
output.  This  is  an  output  side  component. 


Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 


Ef  f iciency 

Operational  Definition  - 

The  ratio  of  expected  input  to  actual  input. 
This  is  an  input  side  component. 


Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 
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Productivity 


Operational  Definition  - 

The  ratio  of  output  to  input  in  the  same 
units. 

Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 


Qual i ty 

Operational  Definition  - 

The  degree  to  which  each  step  of  the  process , 
and  the  process  as  a  whole,  operates 
correctly . 


Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 
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Innovation 


Operational  Definition  - 

An  index  of  the  creative  process  of  changing 
to  respond  to  new  pressures,  opportunities, 
and  threats . 


Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 


Quality  of  Work  Life 

Operational  Definition  - 

The  reaction  of  the  organization’s  personnel 
to  the  work  environment,  including  factors 
such  as  pay,  leadership,  autonomy, 
involvement,  and  relationships. 


Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 
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Budgetabi 1 i ty 


Operational  Definition  - 

The  relationship  of  budgets  and  goals  with 
actual  costs  and  accomplishments.  This 
component  is  analogous  to  profitability  in  a 
private  organization. 


Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 


Effectiveness  and  quality  have  been  suggested  as  the  two 
components  most  critical  to  the  performance  of  a  civil 
engineering  organization.  The  other  components  depend  on 
these  two.  Do  you  agree?  Why  might  you  think  this  to  be 
the  case? 
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Why  is  customer  satisfaction  important  to  the  Engineering 
Branch? 


Who  are  customers  of  the  Engineering  Branch? 


How  does  the  Engineering  Branch  serve  each  of  these 
customers?  How  can  they  be  better  served? 
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PERFORMANCE  COMPONENT  DIAGRAM 


USER  -- 
NEED 


- Budgetabi  1  i  ty 

Productivity - 


Innovation 


--  INPUT 


TRANSFORMATION 


Quality  of 
Work  Life 


OUTPUT USER 

!  SATISFACTION 


Ef  f iciency 


Ef  f ectiveness 


Qual i ty 
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Session  13 


Application  of  Performance  Components 


Objectives:  Apply  the  components  of  performance  to  the 

Engineering  Branch  environment.  Suggest 
elements  of  performance  which  could  be 
targeted  for  evaluation  and  improvement  under 
a  performance  management  framework. 


Critically  analyze  one  of  the  components  of  performance  in 
its  application  in  your  organization. 


What  is  a  reasonable  operational  definition  of  this  aspect 
of  performance?  How  can  it  be  measured  or  evaluated  in  your 
organization? 


Suggest  how  this  would  allow  managers  to  track  performance 
and  determine  where  improvement  is  possible.  How  would  the 
improvement  be  recognized  and  reported? 
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Session  14  Implementation  Details 

Objectives:  Outline  the  steps  involved  in  a  top-down 

implementation  of  a  Performance  Management 
Framework . 

Illustrate  the  technique  of  using  a  pilot 
project  to  initiate  the  program. 

Describe  the  importance  of  the  facilitator  to 
the  operation  of  the  team. 

Describe  how  the  following  need  to  be  addressed  within  the 
context  of  a  performance  management  framework  in  order  to 
promote  successful  improvement  efforts. 

Management  direction 

Strategic  planning 


Identification  of  principal  processes  comprising  work 


Selection  of  process  as  pilot  project  for  first 
improvement  effort 


Selection  of  process  for  next  team  improvement  effort 


Management  support 

Assignment  of  members  to  the  team 


Training  decisions 


Selection  of  a  facilitator  to  assist  team 
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Team  process  definition 

Identify  process  steps 

Determine  input  and  output 

Establish  ownership 

Define  interfaces  with  other  functions 

Team  improvement  efforts 
Identify  problems 

Propose  solutions 

Recommend  actions 

Who  responsible 

Schedule  and  milestones 

How  to  measure  for  success 

Management  implementation 

Screen  implementation  actions 

Support  implementation  actions  selected 

Reward  efforts 
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Session  15 


Implementation  Exercise 


Objectives:  Provide  a  situation  in  which  members  can 

experience  implementation  of  a  performance 
management  framework. 

Discuss  the  lessons  learned  and  how  they 
might  be  applied  to  the  implementation  in  the 
Engineering  Branch  Environment. 

You  are  an  engineering  branch  comprised  of  three  sections, 
programming,  design,  and  construction  management.  Due  to 
the  limited  scope  of  this  exercise,  construction  management 
will  actually  do  the  construction  called  for  in  the 
exercise.  Your  task  is  to  program,  design  and  build  the 
items  identified. 

Part  of  this  task  requires  costing  of  labor  and  materials  to 
be  used.  Costs  are: 

Line  Item  Cost 

Whole  sheets  of  construction  paper  #10  /  sheet 

Unsharpened  pencils  *  5  /  each 

Tape  S  1  /  inch 

Labor  by  builder  #  2  /  minute 

Note  that  sheets  of  paper  are  charged  as  whole  sheets,  even 
if  only  a  portion  is  used.  The  left  over  cannot  be  used  on 
another  project.  Tape  cannot  be  used  in  shorter  than  one 
inch  pieces.  Pencils  cannot  be  broken. 
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What  was  the  hardest  part  of  the  performance  management 
framework  exercise?  Why? 


What  was  the  easiest  part?  Why? 


What  surprised  you  the  most  in  working  through  a 
performance  management  framework? 


Where  do  you  see  the  greatest  hindrance  in  doing  this  in 
your  actual  work?  How  can  this  hindrance  be  overcome? 


What  advantages  and  benefits  do  you  think  can  be  achieved  by 
using  a  performance  management  framework  in  your 
organization? 
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Session  16 


Measurement  Exercise 


Objectives:  Understand  the  importance  of  measurement  in 

evaluating  and  improving  performance. 

List  reasons  for  measuring. 

Define  what  to  measure  as  the  products  which 
pass  between  steps  in  the  process,  from 
internal  suppliers  to  internal  customers. 

Examine  a  method  for  analyzing  variance 
between  these  steps. 

References:  Variance  matrix  -  Total  Quality  Assurance 

Through  Process  Management,  Kenneth  R. 
Jennings  and  others,  AFIT,  1989,  pp  50-57. 

Why  is  measurement  important  in  evaluating  and  improving 
performance  in  your  organization? 


What  uses  can  you  see  for  measurement  of  your  performance? 
How  can  it  help  managers  understand,  control  and  improve  the 
work  of  the  organization? 


What  is  variance'’  What  is  the  relationship  of  variance  and 
measurement? 


What  are  some  measures  of  end  product  variance  that  might  be 
used  in  the  Engineering  Branch? 
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Where  is  variance  located  within  the  work  process? 


How  might  variances  at  earlier  steps  in  the  process  affect 
variances  at  later  steps  in  the  same  process? 


How  can  measuring  and  controlling  variance  at  early  steps 
in  a  complex  process  help  control  the  end  result? 


How  effective  might  improvement  efforts  be  at  this  early 
point  in  the  process? 


Variance  Matrix 

The  variance  matrix  analysis  is  a  tool  that  helps  identify 
the  key  variances  which  affect  subsequent  steps  in  the  same 
process.  The  steps  in  a  variance  matrix  analysis  are  listed 
below.  The  steps  are  then  illustrated  in  an  example  on  the 
following  pages. 

Step  1:  Begin  with  the  list  of  steps  which  make  up  the 
work  process  being  evaluated.  This  includes  every  action 
needed  to  transform  the  input  to  output. 

Step  2:  Group  sets  of  individual  steps  together  into 
clusters.  Each  cluster  should  accomplish  an  identifiable 
change  in  the  product  in  its  transformation  from  input  to 
output.  This  simplifies  the  analysis  by  concentrating  a 
fewer  number  of  main  steps. 

Step  3:  Identify  all  possible  sources  of  variance  in  each 
step.  These  cause  the  work  to  miss  conforming  to  the 
standard  or  norm  set  for  it. 

Step  4:  List  variances  chronologically  in  order  of  how  they 
might  occur  in  the  process,  matching  each  to  the 
corresponding  major  step  in  the  process. 
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Step  5:  Identify  upstream  variances  Ihat  impact  or 
interfere  with  the  control  of  downstream  variances.  A 
variance  matrix  is  used  for  this  purpose.  Check  each 
variance  against  all  downstream  variances  listed  beneath  it. 
If  a  variance  impedes  control  of  a  downstream  variance,  a 
mark  is  made. 

Step  6:  Locate  key  variances,  those  which  impede  control 
over  several  downstream  variances.  The  degree  of  impact  the 
variance  has  directly  on  cost  and  quality  is  also 
considered . 

Step  7:  Establish  the  factors  which  are  important  in 
controlling  these  key  variances. 
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Example  analysis  using  the  variance  matrix. 

The  process  is  the  operation  of  a  machine  which  assembles 
ham  and  cheese  sandwiches. 

Steps  1  and  2  in  the  Variance  Matrix  analysis  are 
illustrated  below.  The  detailed  steps  in  the  process  are 
listed  on  the  right,  and  are  grouped  into  major  steps  as 
shown  on  the  left. 


Clustered  Ma  j_or  Steps 
Prepare  Bread 

Add  ham 

Add  cheese 

Insert  lettuce 

Finish  sandwich 

Package  sandwich 


Actual  Steps 

Replenish  bread  and  mustard 
Fill  machines 

Sort  bread  onto  assembly  tray 

Get  container  of  ham 
Slice  ham 

Position  ham  on  bread 

Get  container  of  cheese 

Slice  cheese 

Position  cheese  on  ham 

Cut  lettuce 

Remove  lumpy  lettuce 

Position  lettuce  on  cheese 

Obtain  bread  for  top 
Position  bread  on  lettuce 
Cut  sandwich 

Insert  sandwich  in  machine 
Insert  cardboard  slip 
Prepare  labels 
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Steps  3  and  4  in  the  Variance  Matrix  analysis  are 
illustrated  below.  Individual  variances  were  determined  and 
then  listed,  clustered  with  the  major  step  they  correspond 
to,  which  are  listed  across  the  top.  Refer,  for  example,  to 
the  major  step  "insert  lettuce".  The  variances  14  and  15 
can  be  seen  listed  under  this  step.  Similarly,  the 
variances  for  each  step  are  listed  beginning  under  the 
appropriate  step,  and  in  order  of  potential  occurrence. 

The  Variance  Matrix 

Prepare  Add  Add  Insert  Finish  Package 

Bread  Ham  Cheese  Lettuce  Sandwich  Sandwich 

II  i  i  i 

il  i  i  t 

1  Bread  soggy 
x2  Edges  curled 
xx3  Bread  crooked 
4  Bottleneck 

5  Ham  too  warm 
6  Slices  too  thick 
7  Ham  shredded 
8  Ham  spoiled 

x  9  Ham  crooked 

10  Cheese  Moldy 
1 1  Cheese  unwrapped 
x  12  Slices  stuck  together 
x  13  Cheese  crooked 

14  Lettuce  lumpy 
15  Lettuce  wet 

16  Run  out  of  top  slices 
17  Slices  wrong  size 


X  X 

X 

10  Top 

not  level 

XX 

X  X 

X 

xl9  Cut 

qual i ty 

X 

XX 

X  X 

20  Package  qual 

XX 

X  XXX 

X 

XXX 

x21  Too  big 

X 

X  XXX 

XX 

XXX 

xx22  Not  sealed 

23  Bad  label 

X  X 

X  X  XX 

X 

X  Key  Var. 

Steps  5  and  6  in  the  Variance  Matrix  analysis  are  also 
illustrated  above.  For  each  variance,  beginning  with  number 
1,  the  list  below  that  variance  was  scanned  for  any 
variances  which  might  be  affected  by  the  preceding  variance. 
Such  relationships  were  marked  with  an  "x".  For  example, 
note  that  variance  15  was  determined  to  possibly  affect 
variances  20  and  22,  while  variance  16  had  no  effect  on 
subsequent  variances.  The  key  variances  were  then  selected, 
and  noted  with  a  "X"  at  the  bottom  row.  These  were  selected 
because  of  their  multiplied  effect  on  other  variances  or 
because  on  their  own  they  were  important  enough  to  be  called 
a  key  variance . 
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Step  7,  not  illustrated  above,  would  then  be  to  determine 
factors  which  control  the  behavior  of  the  key  variances. 
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Session  17  Cause  and  Effect  of  Measurement 

Objectives:  Understand  the  importance  of  measurement,  and 

the  components  of  an  effective  measurement 
system. 

Reference:  The  cause  and  effect  technique  -  DOD  5000.51- 

G ,  Total  Quality  Management x  A  Guide  for 
i on  ,  dated  15  Feb  89,  p  51. 


Major  Cause 

- >  ! 

Minor  ! 
Causes  ! 

- >  : 


- > 

Minor 

Causes 

- > 


Major  Cause 

!  < - 

!  Minor 
!  Causes 

!  < - 

< 

i 

- >  Result 

» 

t 

•  < - 

!  Minor 
S  Causes 

!  < - 


Major  Cause  Major  Cause 

What:  Represents  the  relationship  between  an  effect 
(problem  or  result)  and  its  potential  causes. 

Why:  The  diagram  is  drawn  to  sort  and  relate  the 

interactions  among  the  factors  affecting  the  result. 


How:  1 .  Name  the  problem 

2.  Decide  the  major  categories  of  causes.  Major 
causes  may  include:  data  and  information  systems, 
dollars,  environment,  hardware  and  equipment, 
materials,  measurements,  methods,  people,  and 
training.  Frequently,  the  four  categories  used 
are  people,  machines,  methods,  materials. 

3.  Brainstorm  for  more  detailed  causes. 

4.  Eliminate  causes  that  do  not  apply. 

5.  Discuss  the  remaining  causes  and  decide  which  are 
most  important. 

6.  Work  on  most  important  causes. 
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Eliminate  or  control  causes. 


Cause  and  Effect  Analysis  for  an  Effective  Measurement 
System 


Ef  f ective 
Measurement 
System 


Session  18  Objectives  Matrix 

Objectives:  Demonstrate  a  technique  to  summarize  non- 

related  measures  into  a  composite  score. 

Describe  the  distinction  between  product, 
process  and  surrogate  measures ,  and  the 
advantages  of  each. 

Reference:  Objectives  matrix  -  Production  Systems 

P 1 ann ing A  Analysis^  and  Control,  by  James  L. 
Riggs,  1987,  pp~648-660 . 

Product  Measures  -  Actual  attributes  of  item  ready  for  user 


Process  Measures  -  How  product  was  attained 


Surrogate  Measures  -  Reflect  actual  attributes 


What  are  the  advantages  and  disadvantages  of  each? 


What  precautions  are  needed  when  using  process  and  surrogate 
measures?  How  can  you  check  their  validity? 
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Objectives  Matrix  Worksheet 


The  following  steps  make  up  the  objectives  matrix  analysis. 

1.  Establish  Key  Performance  Areas  (KPAs) 

2.  Determine  Key  Performance  Indicators  (KPIs) 

3.  Generate  transformation  curves 

4.  Assign  relative  weights 

5.  Compute  performance  index 

1.  Establish  KPAs 

KPAs  are  those  performance  components  that  are  considered 
important  to  the  organization’s  success.  Usually  these  are 
determined  by  management  or  reflect  the  policies  and 
objectives  set  by  management.  For  example,  the  design 
branch  may  have  the  following  as  its  KPAs. 

Quality  of  the  transformation  process 

Efficiency  of  manpower  use 

Effectiveness  in  the  form  of  timeliness 

Quality  perceived  by  the  user 

Quality  of  the  drawings  and  specifications 

Productivity 

2.  Determine  KPIs 


Next  a  measurable  characteristic  must  be  found  for  each  of 
these  KPAs.  These  must  be  accurate,  quantifiable,  and 
representative  of  the  performance  component  being  measured. 
For  example,  KPIs  for  the  KPAs  above  might  be: 


Ef  f iciency 

Ef  f  ectiveness 
Quality  of  product 

Quality  to  user 

Quality  to  user 

Quality  of  process 

Product! vi ty 


Direct  Productive  Manhours/ 

Total  Manhours 

*  Value  Late  Projects  x  Days  Late 

*  Errors  found  by  design  chief  at 

100%  complete 

User  Survey  Response,  scaled  1-10 
Post-Design 

User  Survey  Response,  scaled  1-10 
Post-Construction 
Total  t  Change  Orders/Total  * 
Contracts 

*  Value  Designed/Manhours  Used 


3.  Generate  transformation  curves 


This  is  the  principal  distinctive  of  the  objectives  matrix 
analysis.  The  dissimilar  performance  indicator  measures  are 
converted  into  similar  ten  point  index  scores.  These  scores 
can  then  be  combined  as  a  weighted  average  into  an  overall 
performance  index.  The  transformation  curve  is  created 
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individually  for  each  KPI  by  first  determining  the  anchor 
values,  and  then  filling  in  the  intermediate  values.  The 
anchor  values  include  the  zero  and  ten  values  on  the  index. 
Addi tional ly ,  the  value  for  three  or  five  is  fixed  to 
further  anchor  the  index.  For  example,  the  KPI  for  quality 
of  process,  the  ratio  of  change  orders,  would  be  anchored  as 
f ol lows . 

Index  Value  of  KPI  measure 

Value 

.5  (If  change  orders  total  half  of  the 
project  cost,  award  no  points) 

. 12  (Anchor  the  midpoint  at  the  historical 
mean  of  12%) 

0.0  (The  best  possible  is  no  change  orders, 
award  full  points) 


Then  fill  in  the  rest 

o  f 

the 

scale.  This  can  be  done 

in  any 

combination  of  linear 

or 

curved  relationships  as  seems 

appropriate  for  the  measure. 

Graph  the  relationship 

if 

desired  to  make  the  function 

more  understandable. 

Quality  of  Process 

Index 

KPI 

10  * 

10 

0.0 

!  * 

g 

.02 

!  * 

8 

.04 

!  * 

7 

.  065 

!  * 

6 

.09 

s  :  # 

3 

.  12 

!  * 

4 

.  16 

1 

i 

« 

3 

.21 

f 

1 

ft 

2 

.  28 

1 

1 

*  1 

.37 

o  : 

*  0 

.  5 

0  .1  .2  .3  .4  .5 


Zero 

Five 

Ten 
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Now,  fill  in  the  values  for  the  rest  of  the  KPI 
transformation  curves.  Anchor  values  have  been  provided. 
Graph  the  values  if  desired.  Use  any  curved  or  linear 
relationships  desired  to  fill  in  the  values. 


Index 

Ef f ic. 

Effect. 

Qual i ty 
to  user 

Qual i ty 
product 

Prod . 

10 

9 

8 

7 

6 

>  .75 

0 

10 

0 

2 

5 

4 

3 

2 

1 

.  4 

3200 

5 

5 

1 

0 

0 

>  100000 

0 

10 

<  .  5 

4.  Assign  weights 

The  next  step  is  to  weight  each  of  the  KPIs  so  they  can  be 
combined  into  a  weighted  average.  The  sum  of  the  weights 
must  be  one.  Give  more  weight  to  the  KPIs  which  are 
believed  to  be  most  important,  and  which  are  believed  to  be 
most  valid.  For  instance,  in  the  list  above,  the  value  of 
change  orders  is  an  important  measure  of  how  accurate  the 
design  is.  It  is  also  a  highly  valid  measure  because  the 
cost  of  change  orders  can  be  accurately  determined.  This 
KPI  should  be  weighted  more  heavily.  On  the  other  hand,  the 
post-design  user  survey  is  not  as  important.  The  users  often 
cannot  judge  accurately  the  quality  of  the  design;  they  do 
not  have  the  expertise.  This  KPI  should  not  be  weighted  as 
heavily.  You  be  the  judge  on  the  others.  Fill  in  the 
weights  below. 


4 
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Weight 


KPI 

Total  *  Change  Orders/Total  * 
Contracts 

Direct  Productive  Manhours/ 

Total  Manhours 

*  Value  Late  Projects  x  Days  Late 
User  Survey  Response,  scaled  1-10 

Post-Design 

User  Survey  Response,  scaled  1-10 
Post-Construction 

*  Errors  found  by  design  chief  at 

100%  complete 

M  Value  Designed/Manhours  Used 


Sum  1 . 00 


5.  Compute  Performance  Index 

Finally,  the  performance  index  can  be  calculated.  This  is 
done  be  measuring  the  performance  for  each  KPI.  The  measure 
is  converted  into  an  integer  score  between  zero  and  ten 
using  the  transformation  curve.  These  scores  are  then 
averaged  into  the  performance  index  using  the  weights.  This 
procedure  is  made  easy  using  a  table,  below. 

KPA  Proc  Effec  Userl  Prody 

Effic  Prodt  User2  Score 


Raw  Value  . 17 


Transformation  0  >.75  0  10  10 

Curves  . 02 

.04 
.065 
.  09 

.12  .4  3200  5  5 

.  16 
.21 
.  28 
.  37 

.5  0  >100000  0  0 


0  2  10 

9 
8 
7 
6 

5  1  5 

4 
3 
2 
1 

10  < . 5  0 


4 


Score 


4 


Weight 

Weighted 

Score 


Performance  Index 
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Compute  the  performance  index  for  the  following  measured 
KPIs  . 


quality  of  PROCess  .17 

EFFICiency  .66 

EFFECtiveness  5000 

quality  to  USER1 

post-design  6 

quality  to  USER2 

post-construction  3 
quality  of  PRODucT  8 

PRODuotivitY  1.25 


Fill  in  your  transformation  curve  values  from  earlier  in  the 
worksheet.  Use  the  same  values  for  USER1  and  USER2 .  Fill 
in  your  weights  for  the  KPIs  in  the  appropriate  place  in  the 
table.  Enter  each  raw  value  on  the  appropriate  line  near 
the  top  of  the  table.  Find  the  value  on  your  transformation 
curve  which  most  closely  matches  that  value.  Record  this 
value  on  the  line  given.  Multiply  each  score  by  its  weight 
to  obtain  the  weighted  score.  Sum  these  weighted  scores  and 
record  the  result  at  the  bottom  of  the  table  as  your 
performance  index.  This  number  should  be  between  zero  and 
ten.  The  closer  to  ten,  the  better  overall  performance  for 
your  organization. 

Conclusion 

This  objectives  matrix  can  be  used  with  any  measures 
appropriate  for  your  organization.  It  allows  you  to  quickly 
determine  an  overall  ranking  for  your  organization  for  the 
time  period  studied.  This  can  allow  management  to  track  the 
organizations  performance  over  time  and  make  recommendations 
for  improvements  as  needed. 
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What  type  was  each  measure,  process,  product  or  surrogate? 
Why? 


How  might  a  manager  use  such  a  combined  index? 


How  would  such  an  index  be  useful  specifically  for 
improvement? 
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Appendix  Leaden_|_s  Guide 


This  appendix  contains  the  Leader’s  Guide  for  a  three 
day  training  course  in  performance  management.  It  follows 
the  Plan  of  Instruction  given  in  Appendix  A.  It  is  intended 
for  use  by  the  course  leader/instructor  to  guide  the 
discussion  and  activities  planned  to  communicate  the 
material .  Each  student  in  the  course  needs  to  be  provided 
with  a  copy  of  the  Student’s  Handbook  in  Appendix  B.  This 
handbook  follows  the  same  plan  and  provides  the  students 
with  the  outlines  and  information  needed  for  the  course  to 
be  successful . 

An  introduction  and  course  schedule  are  given. 
Objectives  are  given  for  the  course  and  each  session. 
Detailed  instructions  are  given  for  leading  each  session. 
These  instructions  are  superimposed  over  a  copy  of  the 
Student’s  Handbook.  The  format  for  this  is  explained  in  the 
Leader’s  Guide.  The  instructions  are  intended  to  allow  a 
member  of  the  base  level  Civil  Engineering  Squadron  to  lead 
the  course,  even  if  this  individual  has  no  previous 
experience  in  performance  management  or  leading  courses. 

In  preparing  thi3  handbook,  material  was  adapted  from 
other  sources  as  appropriate.  This  material  is  referenced 
be  1 ow: 

Session  2  Case  Descriptions  (Hayes,  1990:19-24). 

Session  3  Management  Cases  (Wertz,  1989; 

Warmington,  1988;  Simmons,  1990:74-76; 

Barra,  1989:46-50) 
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Session 

5 

Group  roles  (HQ  AFLC ,  1989d :  120- 12  1 )  . 
Conflict  responses  (Gray  and  Smeltzer, 
1989:412-413) . 

Session 

6 

Participation  exercise  (HQ  AFLC, 

1989c: 11-12  to  11-16) . 

Session 

9 

Process  improvement  plans  (Jennings  and 
others,  1989:12-89). 

Session 

12 

Performance  components  (Sink  and  Tuttle 
1989: 171-186) . 

Session 

16 

Variance  matrix  (Jennings  and  others, 
1989:50-57) . 

Session 

17 

Cause  and  Effect  (Department,  1989a:51) 

Session 

18 

Objectives  matrix  (Riggs,  1987:648-660) 
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LEADER’S  GUIDE 


PERFORMANCE  MANAGEMENT  FRAMEWORK  HANDBOOK 

A  handbook  for  the  evaluation  and 
improvement  of  performance  in  the  Engineering  Branch  of 
Air  F'irce  Civil  Engineering  Squadrons 

FOREWARD 

This  handbook  is  designed  to  assist  Engineering  Branch 
managers,  engineers,  and  others  who  contribute  to  the 
preparation  and  execution  of  projects  for  real  property 
construction,  maintenance,  and  repair.  These  individuals 
are  challenged  with  providing  and  maintaining  quality 
facilities,  allowing  for  Air  Force  organizations  to  perform 
their  missions.  Evaluation,  management  and  improvement  of 
the  performance  of  Engineering  Branch  personnel  is  central 
to  continuing  to  fulfill  this  mandate. 

4 

The  goal  of  this  handbook  and  the  training  which  accompanies 
it  is  to  acquaint  Engineering  Branch  members  with  a 
framework  for  managing  performance  and  to  provide  the  tools 
needed  to  use  this  framework. 


179 


This  leader’s  guide  provides  the  guidance  needed  to  allow  a 
member  of  the  Engineering  Branch  to  lead  a  group  of  his/her 
peers  through  the  training  course  included.  This  person 
need  not  be  experienced  in  performance  management  or  leading 
courses.  The  leader’s  guide  provides  adequate  instructions 
to  direct  the  leader  through  each  session.  A  short  amount 
of  time  spent  becoming  familiar  with  the  lesson  contents 
before  each  day’s  lessons  will  equip  the  leader  for  the 
role. 


4 
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Performance  Management  Framework  Objectives: 

Foster  a  management  perspective  conducive  to 
performance  improvement 

Create  an  organizational  environment  promoting 
performance  improvement 

Continually  encourage  the  quest  to  discover  and 
implement  ways  to  improve  performance 


Course  Objectives: 

To  equip  civil  engineering  personnel  to  manage  a 
performance  management  framework 

To  create  momentum  for  performance  improvement 

To  promote  within  civil  engineering  personnel  a 
perspective  for  improvement 

To  provide  the  essential  tools  and  techniques  to 
establish  and  maintain  a  performance  management 
framework 

Explanation  of  Format: 

The  lesson  plan  outline  is  included  in  each  lesson. 

The  lesson  headings  and  sub-headings,  together  with  the 
recommended  time  for  each,  are  underlined.  These  do 
not  appear  in  the  student  handbook. 

Notes  for  leading  the  lesson  are  preceded  by  ’##*##* 
and  are  indented.  These  notes  do  not  appear  in  the 
student  handbook. 

All  other  text  appears  in  the  student  handbook  as 
shown . 

General  Instructions: 

Prior  to  each  day’s  training,  the  leader  should  read 
through  the  Leader’s  Guide  for  all  the  lessons  in  the 
day.  Become  sufficiently  acquainted  with  the  material 
and  schedule  of  time  allotted  to  move  the  class  group 
through  each  lesson  smoothly.  Get  a  feel  for  how  each 
lesson  or  subject  transitions  into  the  next.  The 
leader  should  make  notes  in  the  guide  to  help  keep 
organized . 
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The  leader  should  use  a  clean  paper  flip  pad  and  easel 
or  overhead  projector  with  plenty  of  clean  transparency 
film  and  colored  pens.  The  leader  may  either  assign  or 
ask  for  a  group  member  to  be  the  scribe  for  the  group 
on  the  pad  or  overhead  each  session,  or  do  it 
himse 1 f /hersel f .  Such  visible  recording  is  an 
important  team  technique;  make  liberal  use  of  it. 
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CLASS  SCHEDULE 


First  Day  Time  Page 

Morning 

Topic:  Introduction/Engineering 

Environment  Distinctives 

Session  1  Introduction/ 

Distinctives 

Session  2  Benefits  Available 

Topic:  Role  of  Management 

Session  3  Management  Commitment  1  hour  199 

Session  4  Management  Support  1  hour  207 

Lunch  1  hour 

Af  ternoon 

Topic:  Employee  Participation 

Session  5  Team  Participation  1  hour  212 

Session  6  Participation  Exercise  2  hours  218 


1  hour  185 

1  hour  193 
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Second  Day 


Time 


Page 


Morning 

Topic:  Training 

Session  7  Training  Objectives  1  hour  226 

Session  8  Training  Techniques  1  hour  229 

Topic:  Process  Framework 

Session  9  Process  Improvement  1  hour  234 

Session  10  Process  Flow  Exercise  1  hour  241 

Lunch  1  hour 

A f ternoon 

Session  11  Process  Flow-charting  1  hour  243 

Topic:  Target  of  Improvement 

Session  12  Performance  Definition  1  hour  245 

Session  13  Performance  Components  1  hour  252 

Third  Day 

Morning 

Topic:  Framework  Implementation 


Session 

14 

Implementation  Details 

1 

hour 

253 

Session 

15 

Implementation  Exercise 

3 

hours 

255 

Lunch 

1 

hour 

ernoon 

■Jopic : 

Measurement 

Session 

16 

Measurement  Exercise 

1 

hour 

270 

Session 

17 

Cause  and  Effect 
Discussion 

1 

hour 

276 

Session 

18 

Objectives  Matrix 

1 

hour 

280 
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Session 


Introduction/ Engineering  Environment 
Distinct! ves 


Objectives:  Distinguish  the  Performance  Management 

framework  from  other  management  fads  and 
gimmicks  which  have  come  and  gone. 

Establish  that  this  framework  is  one  which 
civil  engineering  personnel  design  themselves 
to  fit  their  own  particular  situations. 

L®sson  Intliodyct ion  5  min 

*##*#Introduce  self  and  briefly  describe  the  course  goals 
listed  on  the  Forward  page.  Refer  students  to  the 
Foreward  page  of  their  handbooks. 

#####Read  over  the  lesson  objectives  above  with  the  group. 

Comment  that  attempts  to  improve  management  are  nothing 
new  in  the  Air  Force,  but  that  the  framework  for 
management  presented  here  is  different.  Point  out  that 
several  elements  of  this  framework  are  likely  to  be 
things  already  being  practiced  in  the  management  of 
this  organization.  Hopefully  this  course  will  expand 
on  these  and  present  some  new  useful  ways  to  view 
management  of  performance. 


Ice  Breaker/  Stage  Setter 

I  Management  Techniques  mi^n 

List  management  techniques  which  the  Air  Force  or  your 

organization  have  tried.  How  well  have  they  worked? 

*#***Ask  the  group  to  list  and  evaluate  other  management 

techniques  they  have  experienced.  This  question  serves 
as  an  icebreaker  for  the  group.  Encourage  interaction 
and  contribution  by  all  members.  As  a  management 
technique  is  mentioned,  ask  if  everyone  is  familiar 
with  it  or  remembers  it.  If  not,  ask  someone  who  does 
to  explain  it.  Urge  as  many  members  will  to  to  give 
their  opinions  of  the  techniques. 

#*###If  the  group  gets  stuck,  prompt  with  ‘management  by 
objectives,'  and  ‘zero  defects.' 

#####Ask  the  group  to  describe  the  management  techniques 
they  use  now,  and  how  well  they  work. 
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Components  of  a 

§yS£®§§iyi  System 


5-jO  min 

What  ape  some  essential  aspects  of  a  successful  management 

f  ramewopk? 

***««Ask  what  aspects  of  these  techniques  have  been 

worthwhile?  What  have  you  seen  in  these  techniques  of 
value? 

####*Ask  what  the  group  believes  should  be  in  a  successful 
management  program. 


^o^klocks  to 

Management  Systems  5  min 

What  are  some  aspects  of  management  techniques  which  have 
not  worked  or  have  hampered  the  effectiveness  of  the 
technique? 

#####Ask  the  group  to  identify  the  harmful  aspects  of 

management  techniques  they  are  familiar  with.  Start 
with  techniques  they  have  tried  in  the  past.  Move  on 
to  what  they  are  using  now. 

###»#Have  the  group  define  the  destructive  elements  of  the 
aspects  identified. 


Course  Overview 

Framework  Objectives  2  min 

****#Preaent  the  course  to  the  group.  Where  possible,  point 
out  where  the  course  and/or  performance  management 
framework  considers  the  essential  components  just 
listed  by  the  group.  Likewise  emphasize  where  the 
course  and/or  framework  avoids  problems  identified. 
Refer  the  group  to  the  objectives  page  at  the  beginning 
of  the  handbook.  Review  the  framework  objectives 
listed  there. 


Course  Objectives 

*####Review  the  course  objectives  listed  on 


2  min 

the  same  page. 
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E rj v  1^ r o n me n t 

Distinctives 


5  min 


**»**Refer  to  the  course  schedule  outlined  at  the  front  of 
the  handbook.  If  desired,  make  overhead  slides  of  the 
schedule  to  direct  the  group’s  attention  to  the  leader. 
Note  that  this  lesson  and  the  one  following  dwell  on 
the  subject  of  the  distinctives  of  the  engineering 
environment.  Present  and  discuss  the  information 
outlined  on  the  slides  attached  to  this  lesson.  The 
slides  do  not  appear  in  the  student  handbook.  The 
first  slide  is  an  introduction.  Each  of  the  following 
three  slides  have  a  question  to  be  discussed.  Use  the 
overhead  or  easel  pad  to  record  the  discussion.  Note 
in  closing  this  section  that  a  performance  management 
framework  is  not  a  'cookbook*  approach  to  performance 
improvement.  It  merely  provides  tools  and  perspectives 
useful  for  personnel  to  design  their  own  plan. 


Ro  1. e  of  Management 
Employee  Participation 
Training 

Process  Framework  5  min 

#*#**Continue  with  the  course  schedule.  Note  that  the 
remaining  sessions  today  deal  with  the  role  and 
responsibilities  of  the  personnel  in  the  organization, 
both  supervisors  and  employees.  Note  that  tomorrow 
morning’s  sessions  concentrate  first  on  the  extent  and 
timing  of  training.  Secondly  the  sessions  key  on  the 
topic  of  understanding  work  through  a  process 
perspective,  and  the  facility  this  perspective  provides 
for  improvement.  Elaborate  on  these  short  descriptions. 
The  leader  will  need  to  have  become  familiar  with  the 
individual  session  objectives  to  clearly  explain  these. 
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Target  of  I mgro veraen t 
Framework  Imglementat i on 
Measurement  5  min 

#####Continue  with  the  course  schedule.  Explain  that 

tomorrow  afternoon’s  sessions  focus  on  the  target  of 
improvement.  The  important  concept  of  user 
satisfaction  will  be  considered.  In  addition, 
performance  and  its  attributes  will  be  discussed. 

Note  that  the  next  morning’s  sessions  include  an 
opportunity  to  work  through  an  implementation  exercise, 
which  will  bring  together  the  points  previously 
studied.  Finally,  the  last  afternoon  will  examine  the 
important  subject  of  measurement.  The  use  of 
measurement  in  planning  and  evaluating  improvement 
efforts  will  be  considered.  Again,  the  leader  needs  to 
be  familiar  with  the  individual  lesson  objectives  and 
elaborate  on  these  short  descriptions. 

Review  Framework  Objectives  1  min 

####»Refer  again  to  the  objectives  page.  Review  the 

framework  objectives.  Explain  that  as  a  result  of  this 
course,  the  members  of  the  group  will  be  prepared  to 
use  the  framework. 

*##»#The  following  four  pages  are  slides  to  be  used  in 

discussing  the  Engineering  Environment  Distinctives 
above . 
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ENGINEERING  ENVIRONMENT  DISTINCTIVES 


Project  Orientation 


Product  of  Ideas 


People  are  Professionals 
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PROJECT  ORIENTATION 


Engineering  projects  are  often  one-time  unique  solutions  to 
problems . 


What  difficulties  does  this  present  for  managing  a  program 
to  evaluate  and  improve  performance? 
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PRODUCT  IS  IDEAS 


Ideas  are  hard  to  quantify  or  evaluate. 


What  is  the  measure  of  a  good  project? 
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PEOPLE  ARE  PROFESSIONALS 


Professionals  are  skeptical  about  a  cookbook  approach  to 
performance  improvement. 


How  can  professionals  be  motivated  to  participate  in  a 
performance  improvement  plan? 
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Session  2 


Potential  Benefits  from  Performance 
Improvement 


Objectives:  Detail  what  benefits  can  be  gained. 

Demonstrate  why  improvement  action  is  needed. 

Understand  benefits  experienced  in  another 
workplace  environment.  Discuss  differences 
in  the  civil  engineering  environment  that  may 
require  adaptation  or  changes  of  approach 
from  that  used  successfully  in  other 
si tuations . 

Reference:  The  three  case  descriptions  -  ’Three  Views  of 

TQM, ’  by  Glenn  E.  Hayes,  Quality,  Volume  29, 
April  1990,  pp  19  -  24. 


Lesson  Plan^.  Introduction  2  min 

#*#*#Go  over  the  session  objectives  above. 

*###«Explain  that  a  good  way  to  understand  a  performance 

management  framework  is  to  look  at  examples  of  its  use. 


Case  Evaluations  cases 23  min 

##*«#Divide  the  group  into  three  teams.  Assign  one  of  the 
cases  on  the  following  pages  to  each  team.  Instruct 
the  teams  to  read,  evaluate,  and  draw  conclusions  from 
the  cases.  The  following  questions  are  to  be  used  in 
forming  their  evaluations.  Inform  them  that  each  team 
is  to  present  a  5-8  minute  summary  to  the  entire 
group.  Tell  them  they  have  20  minutes  to  make  their 
preparation . 
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Questions  for  Case  Evaluations: 


What  need  did  this  organization  identify  which  they 
sought  to  improve? 


What  action  did  they  take  to  make  improvements?  What 
tools  were  used?  What  resources  did  they  tap? 


What  benefits  did  they  achieve?  What  improvements  were 
real i zed? 


How  did  this  environment  differ  from  the  Air  Force 
Engineering  Branch  environment  you  are  in?  In  what 
ways  was  it  similar? 


Would  the  approach  used  work  in  your  organization?  Why 
or  why  not?  What  would  need  to  be  done  differently  to 
make  improvements  to  some  area  of  need  in  your  branch? 
What  could  be  done  in  a  similar  fashion? 
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Case  1.  General  Dynamics  Space  Systems  Div.,  San  Diego,  CA 

In  1985,  senior  managers  at  General  Dynamics  decided  to 
use  performance  management  principles  to  make  their  division 
prosper.  A  quality  team  was  created  to  examine  the 
organization.  Work  processes  were  found  which  were  stuck 
in  the  trap  of  always  doing  things  the  same  without 
questioning  whether  there  was  a  better  way.  An  atmosphere 
was  created  to  encourage  making  changes  without  workers 
fearing  reprisals. 

The  success  of  the  performance  improvement  effort  was 
attributed  to  a  large  degree  to  the  support  and  direction 
provided  by  the  top  management.  This  group  showed 
leadership,  not  just  management,  in  making  sure  the  employee 
improvement  teams  had  all  they  needed  to  tackle  nagging 
problems  and  bring  misunderstood  work  processes  to  light. 

Decisions  were  made  about  training,  including  who, 
what,  when  and  how.  Senior  management  sought  training  in 
how  to  provide  a  more  favorable  environment  in  which  the 
workers  could  proceed  with  the  performance  improvement 
efforts.  Management’s  commitment  was  shown  further  by 
making  sure  everyone  else  also  received  the  training  they 
needed  in  order  to  begin.  Then  management  stood  back  and 
let  teams  of  workers  dive  into  evaluating  and  improving 
their  work  processes  using  performance  management 
techniques . 

As  a  result,  the  team’s  departments  are  now  working 
together  in  better  harmony.  The  material  procurement, 
manufacturing,  production  and  quality  departments  are 
functioning  more  as  a  single  unit.  They  communicate  more 
frequently  and  effectively.  Costs  have  been  reduced  by  as 
much  as  3051  as  well.  Improvements  were  made  not  only  on  the 
factory  floor,  but  in  the  administrative  processes.  Prior 
to  using  performance  management,  their  work  to  produce 
launch  vehicles  and  to  provide  launch  services  was 
understood  only  from  an  engineering  viewpoint.  However,  it 
took  the  new  techniques  and  mindset  of  performance 
management  to  understand  the  work  from  a  process  viewpoint. 
This  allowed  the  process  to  be  charted  and  areas  needing 
improvement  to  be  exposed. 

The  improvements  with  the  largest  potential  impacts 
were  those  focused  not  on  production  floor  but  on  the 
administrative  functions.  Paperwork  processes  were  found  to 
have  the  longest  cycle  times.  Shortening  these  turnaround 
times  proved  more  important  than  the  more  direct  production 
work  processes . 
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Case  2.  TRW  Space  &  Defense  Sector,  Redondo  Beach,  CA 

In  mid-1989,  TRW  set  goals  for  the  sector  into  the  new 
decade  and  chose  performance  management  as  a  way  to  help 
reach  these  goals.  One  reason  given  was  to  meet  their 
customers’  expectations.  A  growing  commitment  to 
performance  management  principles  exists  in  government,  with 
whom  much  of  their  work  is  done.  They  did  not  want  to  fall 
behind  their  competitors,  many  of  whom  have  begun  such 
efforts.  A  second  reason  given  was  the  bureaucratic  nature 
of  their  business.  Work  processes  needed  to  be  improved  to 
reduce  the  time  and  cost  associated  with  this  business 
bureaucracy.  Wasted  steps  were  to  be  cut  out  of  the 
processes,  promising  savings  of  up  to  30%.  Worker 
motivation  would  be  increased  as  individual  responsibility 
was  restored  and  waste  eliminated. 

Top  management  supported  performance  management  by 
promoting  the  mindset  needed  to  look  for  improvement,  not 
just  to  get  by.  Involvement  by  every  manager  was  expected. 
Each  was  to  be  receptive  to  improvement  ideas  and  encourage 
the  efforts  of  their  people.  Further  support  was  provided 
in  the  form  of  resources  needed  to  implement  improvement 
teams  and  their  ideas. 

Performance  management  was  formally  begun  when  all 
senior  managers  attended  training  in  how  to  lead  the  effort. 
Process  action  teams  were  formed  and  training  provided  for 
their  members.  These  teams  were  directed  to  evaluate  and 
propose  improvements  in  the  work  processes  of  the 
organization.  A  goal  of  30%  cost  reduction  over  several 
years  was  set.  Each  department  was  made  responsible  for  a 
portion  of  that  goal  each  year. 

Accomplishments  were  made  in  both  production  oriented 
and  idea  oriented  processes.  Six  processes  that  were 
initially  attacked  were:  1) travel  expense  reports;  2)  Time 
card  processing;  3)  New  hires  and  transfers;  4)  Facilities 
requests;  5)  Performance  measurement  systems;  and 
6)  Administrative  support  requests.  Each  was  marked  by 
bureaucracy,  and  improvements  were  made  by  streamlining  the 
processes  and  removing  areas  prone  to  mistakes. 

The  sector  made  sure  that  improvements  in  schedule, 
performance  or  cost  were  not  made  at  the  expense  of  quality. 
A  focus  was  maintained  of  improving  the  work  processes,  and 
not  just  trying  to  dress  up  the  products  that  result.  A 
strong  customer  involvement  was  developed,  making  sure  the 
work  was  in  harmony  with  customer  expectations. 
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Case  3.  M/A-COM  Government  Systems  Div.,  San  Diego,  CA 

M/A-COM  expanded  its  performance  management  efforts 
in  March,  1989  as  a  result  of  observing  improvements 
previously  achieved  in  a  limited  use  of  the  principles  in 
its  production  area.  Products  were  completed  with  less 
labor,  fewer  defects,  and  lower  inspection  costs.  An 
additional  motivation,  as  a  government  supplier,  was  the 
government’s  requirement  that  all  its  suppliers  use  such 
performance  management  principles. 

Decision  making  authority  was  spread  to  lower  levels  in 
the  organization.  Workers  were  recognized  for  their 
contributions  in  performance  management,  setting  the  example 
that  management  was  behind  such  efforts.  Top  management’s 
active  involvement  in  the  improvement  efforts  was  further 
demonstrated  by  including  the  topic  and  efforts  in  the 
executive  council  agenda  for  their  monthly  meetings. 

Initially,  critical  work  processes  to  be  evaluated  and 
improved  were  defined  by  this  council.  Since  then,  other 
processes  have  been  added  as  a  result  of  suggestions  from 
the  employees. 

Top  management  has  been  involved  in  activities  related 
to  performance  management  to  show  their  support.  Funds  have 
been  budgeted  for  training,  which  has  been  provided  to  all 
performance  improvement  teams.  The  efforts  were  given 
publicity  through  the  organization’s  publications,  where 
teams  were  credited  for  their  accomplishments. 

A  consultant  was  hired  to  train  the  employees  in  the 
tools,  mindset,  and  concepts  of  performance  management.  The 
plan  was  publicized  to  all  personnel.  Teams  have  already 
improved  the  data-handl ing  process,  and  the  employee 
suggestion  policy  is  being  evaluated.  Perhaps  one  of  the 
most  significant  accomplishments  has  been  getting  employees 
to  work  together  in  teams.  Barriers  are  breaking  down  and 
communication  improving  as  they  understand  their  work 
processes  and  why  others  involved  in  them  do  the  things  they 
do.  Not  all  improvements  were  large  ones,  but  as  long  as 
they  are  aimed  in  the  right  direction  they  have  been 
welcomed . 

A  critical  element  in  the  team  problem  solving  was  to 
first  properly  identify  the  process.  This  was  done  using  the 
organization’s  mission  statement  and  process  flow-charting. 
The  urge  to  Jump  the  gun  needed  to  be  controlled.  Only 
after  adequate  training  and  thorough  process  definition 
could  successful  efforts  be  pursued.  Using  the  employees 
in  an  interactive,  participative  team  setting  has  unlocked 
much  potential  for  success. 
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Reports  ( 5  —  8  nun  each) 


20  min 


*####Have  the  groups  present  their  findings. 

ITime  gerrai  tt  i  ng)_  5  min 

I  Needs  for  improvement  in 
this  organization 

What  areas  in  need  of  improvement  can  you  identify  in  your 

organization? 

#####Ask  the  question  above.  If  the  group  is  slow  to 

respond,  prod  with  'What  obstacles  prevent  you  from 
finishing  all  projects  on  time,  in  budget?'  and  'What 
prevents  your  being  able  to  satisfy  the  users  all  the 
time  in  all  projects?' 

*»***Comment  when  a  list  has  been  generated  that  these 

things  may  seem  to  be  beyond  their  control ,  but  that 
that  is  what  a  performance  management  framework  seeks 
to  assist  in,  is  finding  where  control  can  be  gained. 

#*#**If  time  does  not  permit  this  discussion,  ask  the  group 
members  to  consider  the  question  during  the  day. 
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Session  3 


Importance  of  Commitment  by  Top  Management 


Objectives:  Demonstrate  that  management  commitment  is 

crucial . 

Identify  ways  that  management  commitment  can 
be  increased. 

References:  Case  Descriptions  -  Design  and  Implementation 

of  Total  Quality  Management  in  a  Civil 
Engineering  Squadron,  by  1st  Lt  Robert  M. 
Wertz,  1989. 

-  Lessons  Learned  from  the  Implementation  of 
Total  Quality  Management  at  the  Naval 
Aviation  Depots  North  Island^  CA,  by  Jeffery 
Allen  Warmington ,  1988. 

-  "FPL  Wins  the  Deming  Prize,'  by  John 
Simmons,  The  Journal  for  Quality  and 
Participation ,  March  1990 ,  pp  74  -  76. 

-  ‘Motorola’s  Approach  to  Quality,'  by  Ralph 
Barra ,  The  Journal  for  Quality  and 
Participation,  Volume  12,  Number  1,  March 
1989,  pp  46  -  50. 

“  A  Performance  Management  Framework  for 
Civil  fngineering,  by  Robert  M.  Gill,  1990. 


Lesson  Planj_  Introduction  5  min 

**»««Review  the  session  objectives  listed  above. 
«****Ask  the  following  questions. 

Who  is  the  top  management  for  your  organization? 


How  critical  to  your  work  is  their  support? 

the  group  has  trouble  with  this  one,  ask  'What  are 
the  things  that  get  done:  what  management  is  pushing, 
or  what  you  believe  to  be  highest  priority?' 

How  does  your  management  show  what  things  they  support? 

»»***Ask  ‘How  do  they  treat  things  they  do  not  hold  to  be 
important? ' 
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Case  Evaluations  ^5  cases )_  20  min 

*****Announce  that  cases  will  be  evaluated  to  observe 

examples  of  management  commitment.  Divide  the  group 
into  pairs.  If  there  is  an  odd  number  of  persons,  have 
one  group  of  three.  Assign  one  case  to  each  team.  If 
there  are  less  than  5  teams,  do  not  use  all  the  cases. 
If  there  are  more  than  5,  assign  some  cases  more  than 
once.  Instruct  them  to  read,  evaluate,  and  draw 
conclusions  about  the  cases.  Tell  them  to  use  the 
following  questions  as  a  guide.  Inform  them  they  will 
give  3-5  minute  briefings  of  their  evaluation  to  the 
entire  group.  Tell  them  they  have  20  minutes  to  make 
their  evaluations. 

Questions  for  Case  Evaluations: 

How  committed  was  top  management  in  this  case  to  the 
task  the  organization  was  working  on?  How  did 
management  show  their  level  of  commitment? 


What  would  you  suppose  management  was  committed  to? 


How  could  management’s  commitment  have  been  increased? 
Suggest  some  specific  ways. 


Who  could  have  the  possible  impact  cn  top  management  to 
increase  their  commitment?  Who  could  convince  them  to 
more  fully  support  the  task  of  the  organization? 
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Case  1.  2750  CES/DEM,  Wr ight-Patterson  AFB ,  OH 


Early  in  fiscal  year  1989,  the  Operations  and 
Maintenance  Branch  of  the  2750  Civil  Engineering  Squadron 
began  a  formal  performance  management  effort.  The  squadron 
is  a  part  of  Air  Force  Logistics  Command,  which  at  the 
direction  of  its  Commander,  was  implementing  Total  Quality 
Management  principles  command  wide.  The  squadron  had 
established  a  quality  committee,  comprised  of  many  of  the 
top  officers  and  supervisors.  This  council  was  chaired  by 
the  Deputy  Base  Civil  Engineer,  and  oversaw  the  quality 
efforts  of  the  squadron.  The  Industrial  Engineering  Branch 
of  the  squadron  had  several  enthusiastic  and  capable  members 
who  were  eager  to  begin  performance  management  efforts 
squadron  wide. 

The  quality  committee’s  job  was  to  promote  improvement 
within  the  organization  and  to  determine  when  to  form 
process  action  teams  and  which  processes  should  be  assigned 
for  analysis.  Committee  members  attended  team  meetings  on 
occasion  to  show  support  and  observe  the  activity.  The 
committee  designated  a  large  conference  room  for  team 
meetings.  They  further  allowed  team  members  time  to  meet, 
usually  an  hour  per  week.  When  a  team  arrived  at  proposed 
solutions  for  improvements,  the  committee  was  briefed  by  the 
team.  The  committee  then  took  action  on  these 
recommendations . 

As  the  effort  progressed,  and  teams  which  were  formed 
had  been  meeting  for  some  time,  the  quality  committee  was 
found  to  provide  inadequate  direction  to  the  process 
improvement  efforts.  One  employee  expressed  a  feeling  that 
were  it  not  for  the  AFLC  commander’s  personally  showing 
interest  and  involvement,  the  input  from  the  Civil 
Engineering  Squadron's  top  management  would  be  almost 
nothing.  It  was  only  because  the  Industrial  Engineering 
Branch  advisors’  efforts,  covering  up  for  lack  of  top 
squadron  support,  that  any  successful  results  were  obtained. 


201 


Case  2.  Naval  Aviation  Depot,  North  Island,  CA. 

In  1984,  management  at  the  Naval  Aviation  Depot  decided 
to  begin  using  performance  management  principles  to  improve 
their  work  processes.  Initial  training  was  slated  for  only 
the  upper  and  middle  management  personnel .  No  training  was 
provided  for  the  workers  and  their  immediate  supervisors. 

The  only  training  these  employees  received  was  that  which  was 
passed  down  by  upper  management. 

The  top  management  at  the  depot  delegated  the  control 
of  the  performance  management  effort  to  a  staff  group,  while 
the  attention  of  top  management  was  then  diverted  elsewhere. 
This  sent  the  signal  to  the  rest  of  the  organization  that 
the  new  improvement  plans  were  not  a  top  priority,  and  that 
top  management  was  not  supporting  the  performance  management 
effort. 

Process  action  teams  were  formed  throughout  the 
organization.  In  fact,  the  explosion  in  the  number  of  new 
teams  being  formed  grew  too  fast  for  upper  management  to 
support  them  effectively.  Not  enough  planning  or  control 
had  been  established  to  direct  the  performance  improvement 
momentum  once  it  had  been  established.  In  spite  of  the 
organizational  time  and  resources  going  into  performance 
management,  pressure  was  still  being  applied  from  users  of 
the  depot  for  all  the  existing  quotas  to  be  met.  Top 
management  was  not  supporting  the  teams  by  coordinating  new 
schedules  of  delivery  with  these  users,  allowing  the  teams 
time  to  get  established. 

The  employees’  performance  appraisals  were  not  changed 
to  reflect  the  new  responsibilities  of  the  process 
improvement  teams.  Soon  the  effectiveness  of  the  effort 
began  to  decrease.  Without  support  from  top  management  for 
a  change  in  emphasis  for  the  organization,  interest  in 
working  for  improvements  trailed  off. 
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Case  3.  Florida  Power  and  Light 

In  1981,  Florida  Power  and  Light  began  using  quality 
teams  to  improve  their  delivery  of  electrical  power  to  its 
customers.  They  achieved  impressive  results,  being  honored 
internationally  by  winning  the  Deming  Prize,  awarded  to  the 
organization  using  performance  management  techniques  best. 
The  transformation  at  Florida  Power  and  Light  was  led  all 
the  way  by  its  top  management.  The  senior  management  team, 
led  by  the  company’s  chairman,  showed  their  commitment  to 
new  ways  of  managing  by  their  behavior  and  their  decisions. 

The  senior  management  team  journeyed  to  Tokyo,  Japan, 
to  discuss  performance  improvement  issues  with  a  team  of 
counselors  there.  Most  any  week  of  the  year  one  of  these 
Japanese  counselors  was  likely  to  be  found  in  Florida 
working  with  some  unit  of  the  organization.  They  worked  to 
install  a  management  system  similar  to  what  utility 
companies  in  Japan  had  found  successful.  The  system 
emphasized  getting  ideas  for  improvements  from  the  workers 
to  management  and  the  other  way  around,  too. 

Each  senior  manager  is  assigned  objectives  for 
improvement  across  several  departments,  to  substitute 
cooperation  for  conflict.  These  managers  made  customer 
satisfaction  their  goal,  and  pushed  this  goal  down  through 
the  levels  of  management  in  the  organization. 

Management  decided  to  enter  the  competition  for  the 
Deming  Prize  as  an  incentive  for  their  company  to  work  for 
even  greater  improvements.  The  organization  already  had 
several  years  experience  in  performance  management  by  this 
time.  Their  efforts  accelerated  as  each  member,  already 
familiar  with  the  performance  management  principles, 
multiplied  their  involvement  toward  this  end.  As  a  result, 
they  accomplished  in  one  year  what  it  might  have  taken  three 
or  four  to  accomplish  otherwise. 

Senior  management  was  committed  throughout,  going  to 
classes,  and  refusing  to  delegate  the  reviews  of  progress, 
but  rather  performed  these  themselves.  They  had  the 
patience  not  the  abandon  the  plan  when  quick  results  did  not 
materialize.  They  remained  dedicated  to  providing  the 
tremendous  investment  needed  to  train  all  employees. 
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Case  4.  Motorola,  Inc. 

Motorola  began  their  performance  management  effort  in 
1980.  They  have  emphasized  a  top-down  plan  and  a  bottom-up 
dedication  of  employees  to  that  plan.  Their  first  step  was 
to  form  a  top-management  council  of  quality  executives. 

This  council  considered  how  to  employ  their  most  valuable 
resource  for  performance  improvement,  their  people. 

An  ambitious  training  program  was  instituted  to  speed 
up  the  improvement  process.  This  included  not  only 
analytical  tools  but  quality  awareness  training  as  well.  In 
addition,  hiring  criteria  was  changed  to  bring  in  more 
quality  conscious  people. 

Motorola’s  twice-a-quarter  policy  and  operating 
committee  meetings  focusing  on  performance  management  were 
chaired  by  no  less  than  the  Chief  Executive  Officer  of  the 
organization.  Progress  in  improvement  initiatives  was 
reviewed,  and  reports  made  by  each  manager  of  a  major  unit 
regarding  the  quality  efforts  of  that  unit.  Successes  as 
well  as  failures  are  briefed.  In  addition,  each  business 
unit  has  its  own  individual  performance  meeting,  again 
twice-a  quarter,  and  again  chaired  by  the  CEO,  to  look  in 
more  detail  at  that  unit’s  efforts. 

Performance  management  planning  i3  not  limited  to  the 
top  officers  of  the  company.  A  program  for  participative 
management  encourages  every  employee  to  contribute 
suggestions  to  teams,  which  then  make  recommendations  for 
improvements.  This  fosters  management  support  and 
communication.  Managers  share  information,  problems  and 
opportunities  with  the  employees,  and  ask  for  their  ideas. 

Management  arranged  for  financial  savings  from  the 
improvement  efforts  to  be  shared  with  the  employees  as  part 
of  an  incentive  plan.  In  addition,  CEO  awards  are  are 
issued  to  recognize  large  achievements. 
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Case  5.  2750  CES/DEE,  Wr  ight-Patterson  AFB ,  OH 


At  the  beginning  of  the  1990  fiscal  year,  the  chief  of 
the  Engineering  Branch  of  the  2750  Civil  Engineering 
Squadron  decided  to  use  performance  management  in  the 
organization  to  solve  a  persistent  problem  with  the  design 
review  process.  Performance  management  principles  were 
chosen  because  of  growing  pressure  from  the  Air  Force 
Logistics  Command  to  see  organizations  turn  to  these 
techniques,  and  to  process  action  teams,  to  improve  work 
processes.  Support  was  provided  by  an  individual  in  the 
Industrial  Engineering  Branch.  Initial  training  was 
provided  to  all  team  members  by  this  individual. 

The  Engineering  Branch  chief  personally  picked  a  team 
of  individuals  from  the  branch  to  participate  in  the  team 
improvement  effort.  The  chief  decided  to  personally  be  a 
member  of  the  team  as  well.  The  chief’s  office  was  used  for 
meetings,  and  the  team  members  were  directed  to  take  time 
out  of  their  schedules  to  attend  and  contribute.  The  chief 
demonstrated  by  personal  attendance,  interest  and 
involvement  in  the  discussions  of  the  process,  that 
management  commitment  was  present.  Willingness  to 
immediately  implement  the  recommendations  of  the  team 
further  showed  this  commitment. 

The  chief’s  influence  was  used  as  needed  to  bring  input 
or  attendance  from  representatives  of  other  organizations  to 
meetings.  This  allowed  the  team  to  get  at  the  root  of  the 
problems  they  encountered.  Memos  were  drafted  and  meetings 
set  up  to  communicate  the  results  of  the  team’s 
investigation.  The  team  members  seemed  to  put  even  more 
energy  into  evaluating  and  improving  the  design  review 
process  after  seeing  their  chief  take  such  an  enthusiastic 
interest  in  the  whole  business. 

When  the  team  had  reached  their  conclusions,  an 
opportunity  was  arranged  to  brief  the  squadron  quality 
committee  on  the  results.  This  committee  was  made  up  of 
many  of  the  senior  officers  and  supervisors  from  the 
squadron.  However,  at  the  appointed  time,  most  all  the 
members  of  the  committee  were  tied  up  elsewhere,  and  the 
presentation  of  results  to  management  was  of  little  impact. 
Nevertheless,  the  Engineering  Branch  chief  continued  to  do 
all  possible  to  see  that  the  team’s  efforts  were 
implemented . 
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Regorts  min  ^ 


20  min 


««»««Have  groups  present  their  findings. 


^PEli£§t i on  5  min 

-  Commitment  of  tog  management 
for  this  organization 

»»«*#Have  the  group  discuss  the  following  questions. 


How  committed  is  your  top  management  to  a  performance 
management  framework? 


How  will  this  level  of  support  impact  your  ability  to  use 
a  performance  management  framework? 


How  can  commitment  by  management  be  increased,  if  needed,  or 
maintained  if  already  adequate?  Who  can  best  convince  your 
top  management  in  this  regard? 
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Session  4 


Management  Support  and  Cultural  Change 


Objectives:  Identify  the  support  roles  of  management  in 

implementing  and  maintaining  a  Performance 
Management  Framework. 

Define  the  concept  of  "Cultural  Change’  and 
communicate  the  essential  role  of  top 
management  in  creating  the  change  in  culture 
from  the  top  down. 

Lesson  Plan^.  Review  and  Introduction  3  min 

»****Comment  that  last  hour  we  looked  at  the  importance  of 
management  support  and  ways  to  increase  that  support. 
Explain  that  this  hour  we  will  build  on  this  basis. 

Review  the  session  objectives  above. 

«»**«Work  through  the  following  structured  discussion  with 
the  group.  Suggested  times  for  each  sub-heading  are 
underlined.  Ask  for  practical  applications,  insights 
and  ideas  of  ways  each  point  is  important.  Make 
overhead  slides  out  of  the  student’s  handbook  or  copy 
the  outline  onto  an  easel  pad  if  desired.  Pause  at  the 
end  of  each  major  heading  to  allow  response/reaction. 

Management  Support  Roles 

Provide  Example  5  min 

Show  interest 

*«**«Genuine  interest,  not  Just  lip  service. 

Get  involved 

Encourage  individual  applications 

*#**#Management  by  walking  around  is  a 

management  style  where  the  boss  goes  to 
the  workplace  and,  without  meddling, 
sees  what  his/her  people  are  doing.  Would 
this  help? 

Apply  the  framework  at  the  top 

#*###Like  the  White  House  not  having  a 
recycling  program  while  preaching 
environmental  issues,  often  it  is  not 
what  we  say,  but  what  we  do. 
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Make  visible  changes 

##*##Show  it  is  not  business  as  usual. 
Provide  Resources  3  min 

Time 


###*#People  need  to  be  free  for  training  and 
evaluation  of  performance. 


Space 

#**##For  meetings. 

Funds 

#####To  implement  improvements  needed. 

Computer  Resources 

*««KftFor  tracking  performance. 

Support  from  consul tants /DEI 

»«**#For  guidance  and  help  in  using  the  tools  of 
a  performance  management  framework. 

Coordinate  Organization-Wide  Efforts  5  min 

Mul t i -discipl ine  applications 

###*#Many  improvement  efforts  will  cut  across 
functional  and  section  or  branch  lines. 

Upper  management  coordination  is  needed  to 
assure  cooperation. 

Planning 

###**Lack  of  resources  may  constrain  applications. 
Planning  is  needed  to  give  the  most  promising 
projects  first  cut  at  the  resources. 

Assure  efforts  are  being  made 

##*##Upper  management  can  track  the  progress  of 
the  improvement  efforts  across  the 
organization.  If  one  area  is  not  making 
progress,  attention  can  be  given  to  see  if 
additional  training  or  resources  are  needed. 
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Select  processes  for  improvement  efforts 

#***#Management  must  determine  which  work 

processes  to  focus  on  first,  and  select  team 
members  to  work  each  process . 

Reward  Results  2  min 

Recognition 

#*###Letters ,  presentations,  or  a  bulletin  board 
specifically  set  up  to  bring  attention  to 
outstanding  performance  management  framework 
achievements . 

Awards 

####*Any  form  of  individual  incentive  awards. 

Access  to  resources 

#»###Reward  high  achievement  sections  with  some 

kind  of  desirable  improvement  project,  or  new 
system  of  some  type  they  can  use. 


Cultural  Change 
Culture 

*####Define  culture  as  the  actual  guidelines  which  an 
organization  follows  in  doing  its  work.  Remark 
that  this  will  become  clearer  through  an  exercise. 

Your  Organization’s  Culture  -  Man  from  Mars 
Perspective  20  min 

##»##If  a  man  from  Mars  were  to  invade  this 

organization,  and  observe  it,  what  would  he 
conclude  is  the  actual  rules  of  play  that  are 
followed.  List  as  many  as  come  to  mind. 

This  is  not  the  official  organizational  goals 
or  charter.  What  would  this  alien  outsider 
conclude  is  the  real  motivators  of  the  work 
place . 

stuck,  prompt  with  'the  squeaky  wheel  gets 
the  grease,'  "put  out  fires,'  'micro- 
management  from  the  top,'  'short  term 
results. * 

###»#Note  that  this  is  the  culture  of  this 
organization . 
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Unneeded  cultural  attributes 


#*###Have  the  group  identify  any  of  the  above  list 
which  are  not  productive,  or  which  could  be 
changed  to  be  more  productive.  Ask  how  this 
would  help. 


Change  3  min 

##*##Note  that  in  order  to  create  a  cultural  change, 
one  must  first  understand  the  present  culture. 

Only  then  can  one  take  control  of  the  culture,  and 
alter  aspects  to  a  more  favorable  environment  for 
improvement.  Comment  that  there  are  two  channels 
through  which  the  cultural  changes  must  be  made. 

Formal  Channels  3  min 

«####This  is  the  official  management  making  and 

communicating  decisions  through  the  official 
chain  of  command. 

Make  new  policy 

Counter  old  culture 

**##*Show  that  the  old  cultural  anchors  are 
no  longer  attached,  that  now  you  are 
marching  to  the  beat  of  a  different 
drummer . 

Informal  Channels  4  min 

*»*ttttThe  informal  group  structure  can  often  carry 
more  influence  with  individuals  than  the 
official  chain  of  command.  This  influence  is 
communicated  through  group  norms  and 
approval . 

Identify  informal  group  leaders 

#####Those  people  who  are  respected  and  whom 
people  take  their  cues  from. 

Solicit  their  assistance 

##*##If  brought  into  your  camp  from  the 

beginning,  these  people  can  be  powerful 
allies.  If  alienated  or  excluded  from 
the  beginning,  these  people  can  block 
everything  you  try  to  do. 
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Conclusion  and  Summary  2  min 

*****Summarize  the  management  roles. 

Provide  Example 
Provide  Resources 
Coordinate  Organization-Wide 
Ef  f  orts 

Reward  Results 

##**«Summarize  Cultural  Change. 

Know  your  organizations  culture. 
Harness  both  formal  and  informal 
channels  to  change  this  culture  to 
better  support  the  performance 
improvement  effort. 
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Session  5 
Ob j  ecti ves : 


References : 


Teamwork  and  Participation  Concepts 

Establish  the  ability  of  a  team  to  do  more 
than  the  individual  can  in  the  subjective 
area  of  performance  improvement  planning. 

Identify  the  types  of  groups:  work  area 
teams  or  multi-functional  cross  disciplinary 
teams.  Discuss  the  effect  of  group  size  on 
group  dynamics. 

Present  common  problems  plaguing  groups. 
Teach  team  members  to  expect  them,  identify 
them,  and  correct  them. 

Group  role  definitions  -  Student^s  Guidex 
AFLC  Facilitator  Training,  1st  Edition, 
October  1989,  pp  120-121. 

Conflict  response  strategies  -  Management^ 
The  Competitive  Edge,  by  Edmund  R.  Gray  and 
Larry  R.  Smeltzer,  1989,  pp  412-413. 


Lesson  Plan^  Introduction  3  min 

###*#Remind  the  group  that  this  morning,  elements  of  the 

engineering  environment  which  make  it  distinctive  for 
performance  management  were  presented.  Management's 
important  role  in  initiating  and  sustaining  the 
improvement  effort  were  discussed.  This  afternoon,  the 
topic  will  shift  to  the  part  employees  and  middle 
managers  play  in  performance  management  framework. 

**»*ttReview  the  session  objectives  above. 


Group  Decision  Making 

Compared  to  Individual  12  min 

###»#Instruct  the  group  to  identify  and  list  some  advantages 
and  disadvantages  associated  with  individual  and  group 
decision  making. 

*»»#*When  adequate  advantages  and  disadvantages  have  been 
listed,  ask  for  situational  factors  which  would  favor 
one  approach  over  the  other.  Many  of  these  will  be 
suggested  from  the  advantages  and  disadvantages.  Some 
examples  are:  speed,  accuracy,  acceptance  of  decision 
by  the  organization,  risk  taking,  and  innovation. 
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»*#*#When  some  situational  factors  have  been  listed,  ask  for 
specific  application  opportunities  in  their 
organization.  What  are  some  decision  opportunities 
where  factors  favor  one  or  the  other,  and  why? 

#####If  the  group  is  stuck  here,  suggest  some  decision 

opportunities  and  ask  which  approach  would  be  favored 
and  why?  Examples  are:  determining  which  projects 
should  be  designed  by  A-E  or  in-house,  deciding 
architectural  treatment  for  a  building,  selecting 
performance  period  for  a  project,  determining  if  a 
contractor’s  question  is  covered  by  the  specifications. 

#*###Finally ,  ask  which  method  would  be  appropriate  for 
performance  improvement  planning.  Evaluating  your 
organization  for  areas  to  be  improved  and  planning 
those  improvements  are  decisions  to  be  made.  What 
advantages  and  disadvantages  for  team  or  individual 
decision  making  can  the  group  suggest.  Summarize  after 
discussion  that  the  complexity  and  importance  of  the 
decisions  indicates  a  team  approach  shows  most  promise. 

Individual  Decision  Making  Group  Decision  Making 

Advantages  Advantages 


Disadvantages  Disadvantages 


Situational  Factors  Situational  Factors 


Application  Opportunities  Application  Opportunities 


Group  Structure 

Composition  8  min 

#»###Explain  that  a  functional-area  group  is  one  composed 
from  members  of  the  same  job  shop  or  work  area.  The 
members  are  the  workers  and  their  immediate  supervisor 
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When  might  a  f unctional -area  group  be  appropriate? 

##»»»Suggest  to  the  group  that  a  functional -area  group 

should  include  everyone  the  the  shop  or  area.  Ask  the 
group  why  this  might  be  important. 

*##**Ask  the  group  what  type  of  goals  such  a  group  would 
have  . 

###»#Ask  what  length  of  time  would  they  meet  over.  Relate 
this  to  the  group  goals. 


#*###Explain  that  a  cross-functional  group,  by  contrast,  is 
made  up  of  representatives  of  each  function  that  is 
involved  in  the  task  concerned. 

When  might  a  cross-functional  group  be  preferred? 

#####Ask  the  group  to  describe  the  type  of  individuals  who 
should  make  up  this  group.  Would  they  be  the  most 
capable  from  each  section  or  the  least?  Why  might  you 
want  the  best  people? 

####*Note  that  these  type  groups  are  most  often  formed  to 
address  specific  problems  in  the  way  things  are  done. 
Ask  what  length  of  time  they  might  continue  m%gting 
f  or . 

#####Ask  the  group  to  contrast  the  two  groups,  noting 
differences  in  their  style  and  purpose. 


Size  7  min 

How  does  group  size  affect  group  effectiveness? 

###»*Comment  that  another  important  aspect  of  group  decision 
making  is  the  group  size.  Ask  the  group  to  discuss  the 
impact  of  having  too  large  a  group.  Too  small. 

*#*##Ask  the  group  what  they  think  of  the  size  of  this  class 
for  group  decision  making. 

####*Note  that  organizational  behaviorists  suggest  6  to  12 

as  a  good  range.  Ask  if  any  one  can  think  of  a  classic 
example  of  group  decision  making  with  groups  this  size. 
Prompt  with  jury  trial  only  if  needed.  Ask  why  it  is 
critical  in  that  situation  to  get  a  good  decision. 
Relate  this  to  the  importance  in  your  organization  to 
get  good  improvement  effort  decisions. 
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Qroug  Roles 


12  5}lH 


Common  Group  Roles: 

#####Read  through  the  following  group  role  descriptions. 

Task  Related: 

Initiator  -  Proposes  tasks  or  goals;  defines  a  group 

problem;  suggests  a  procedure  or  idea  for  solving 
a  problem 

Seeker  -  Requests  facts;  seeks  relevant  information 
about  group  concern;  asks  for  expressions  of 
feelings;  requests  a  statement  or  estimate; 
solicits  expressions  of  value;  seeks  suggestions 
and  ideas 

Giver  -  Offers  facts;  provides  relevant  information 
about  group  concern;  states  a  belief  about  a 
matter  before  the  group,  giving  suggestions  and 
ideas 

Summarizer  -  Interprets  ideas  or  suggestions;  clears  up 
confusion;  defines  terms;  indicates  alternatives 
and  issues  before  the  group;  pulls  together 
related  ideas;  restates  suggestions  after  the 
group  has  discussed  them,  offering  a  decision  or 
conclusion  for  the  group  to  accept  or  reject;  asks 
to  see  if  group  is  nearing  a  decision 

Maintenance  Related: 

Harmonizer  -  Attempts  to  reconcile  disagreements; 
reduces  tension;  gets  people  to  explore 
di f  f  erences 

Gate  Keeper  -  Helps  keep  communication  channels  open; 

facilitates  the  participation  of  others;  suggests 
procedures  that  permit  sharing  remarks 

Encourager  -  Acts  friendly,  warm,  and  responsive  to 
others;  indicates  acceptance  of  others’ 
contributions 

Compromiser  -  Offers  a  compromise;  admits  error; 

modifies  own  ideas  in  interest  of  group  cohesion 
or  growth 
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Standard  Tester  -  Tests  whether  group  is  satisfied  with 
its  procedures;  points  out  explicit  or  implicit 
norms  which  have  been  set  to  make  them  available 
for  testing 

Self  Related: 

Aggressor  -  Deflates  the  status  of  others;  disapproves 
values,  attacks  the  group  problem;  jokes 
aggressively;  seeks  recognition 

Blocker  -  Tends  to  be  negative  and  stubbornly 

resistant;  disagrees  and  opposes  beyond  reason; 
attempts  to  bring  back  issue  after  group  has 
rejected  it;  refuses  to  or  ceases  to  participate 

Dominator  -  Tries  to  assert  authority  in  manipulating 
the  group  or  certain  members;  flatters;  asserts  a 
superior  right  to  attention 

Playboy  -  Makes  a  display  of  his/her  lack  of  ability 
and  involvement  in  the  group  process;  cynical  or 
nonchalant;  engages  in  horseplay 

####*Ask  the  group  if  they  think  all  these  roles  are  useful 
in  group  dynamics.  Why  or  why  not? 

####*Note  that  these  roles  are  not  necessarily  exclusive  to 
one  person,  nor  are  persons  limited  to  only  one  role. 
People  may  adopt  various  roles  as  the  need  arises. 
However,  some  roles  begin  to  be  expected  of  the 
certain  people 
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**###Lead  the  group  through  the  following  discussion. 

What  are  the  merits  and/or  problems  with  each  of  the 
following  ways  of  dealing  with  conflict? 

Separate  the  conflicting  parties 


Impose  rules  or  regulations  to  reduce  the  conflict 


Bring  the  parties  together  to  confront  the  Issues  and 
work  out  solutions 


Session  6 
Objectives : 


Reference : 


Team  Participation  Exercise 

Allow  the  group  members  to  work  as  a  team. 
Give  the  individuals  an  opportunity  to 
observe  group  interaction  techniques  and 
problems . 

Discuss  the  dynamics  of  the  group  exercise. 

Identify  specific  roles  that  people  adopt  in 
a  group  environment  to  facilitate 
performance . 

Discover  the  role  of  a  facilitator  for  the 
group . 

Participation  Excercise  -  Instructor  Guide^ 
AFLC  Facilitator  Training,  1st  Edition, 
October  19897  ll-12~to  11-16. 


Lesson  Plan^  Part icipation  Exercise  50  min 

**»*»Have  the  members  of  the  group  put  away  their  handbooks 
and  other  materials  for  this  group  exercise.  Divide 
the  group  into  at  least  two  teams,  preferably  with  six 
3tuden^  in  each  team.  If  there  are  les3  than  six  on  a 
team  some  members  will  need  to  receive  more  than  one 
clue  card  described  below. 

*»###Say:  Each  member  of  your  team  will  receive  written  bits 
of  information.  These  are  not  to  be  shown  to  others. 
What  will  be  required  of  you,  and  how  to  go  about  it, 
will  become  clear  to  you  as  you  share  information  with 
each  other  through  verbal  communication  only.  When  a 
team  feels  that  the  required  tasks  have  been  completed, 
call  the  judge  (leader)  to  check  your  results.  If  your 
tasks  have  been  only  partially  completed,  or  if  you 
have  done  more  than  required,  the  judge  will  consider 
the  tasks  as  oeing  totally  incomplete.  The  judge  will 
not  share  with  you  what,  if  anything,  has  been 
completed  correctly. 

» # # » #Cont inue :  The  following  rules  will  be  observed 
throughout  this  activity: 

a.  From  the  moment  the  team  begins  work,  members 
may  speak  only  to  other  members  of  their  team. 

b.  You  may  not  show  others  the  content  of  your 
written  bits  of  information. 

c.  You  may  not  write  anything. 

d.  You  must  obey  the  judge’s  instructions. 

You  will  have  the  rest  of  this  hour  to  finish  the  task. 
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*  #  * # ^Separate  the  teams  as  much  as  possible.  Pass  out  the 
following  bits  of  information.  Ask  for  any  questions. 
Have  them  begin. 

###**When  a  team  presents  their  solution,  compare  it  to  the 
answer.  If  not  entirely  correct,  only  say  "incorrect*. 
If  correct,  inform  the  team  when  to  be  back  for  the 
next  session. 

FARMERS  BITS  OF  INFORMATION  SHEET  (A) 

The  dog’s  owner  lives  next  door  to  the  house  with  the  plum 
orchard 

Hull  raises  albino  rats 

The  farmer  who  lives  next  to  Pavlov  drives  a  station  wagon 
Only  one  of  the  village  houses  is  located  on  the  east  side 

FARMERS  BITS  OF  INFORMATION  SHEET  (B) 

Pavlov's  neighbor  raises  chimpanzees 
Skinner  lives  next  to  the  red  brick  house 
The  farmer  who  raises  dog3  also  grows  cherries 
One  of  your  group’s  tasks  is  to  decide  who  drives  a  truck 
The  houses  of  the  village  are  standing  in  a  semicircle, 
beside  each  other 

FARMERS  BITS  OF  INFORMATION  SHEET  (C)  ^ 

Kohler  grows  pears 

There  is  a  limousine  in  the  garage  of  the  ranch  house 
Each  farmer  raises  a  different  kind  of  animal 
Farmer  Thorndike  lives  next  to  farmer  Skinner 
A  motorcyle  stands  in  the  back  yard  of  the  log  cabin 
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FARMERS  BITS  OF  INFORMATION  SHEET  (D) 

The  person  who  raises  cats  lives  next  door,  to  the  east,  of 
the  house  with  almond  trees 
Your  group  has  less  than  three  tasks 

Every  week,  boxes  of  dog  food  are  placed  at  the  gate  of  the 
log  cabin 

Only  one  of  the  village  houses  is  located  on  the  West  side 
Each  of  the  five  farmers  living  in  the  village  drives  a 
different  kind  of  vehicle 

FARMERS  BITS  OF  INFORMATION  SHEET  (E) 

The  log  cabin  is  in  the  most  northern  position  of  the 
vi 1 lage 

Each  farmer  grows  a  different  kind  of  fruit 
The  ranch  house  stands  next  to  the  cottage 
Farmer  Thorndike  drives  a  sports  car 
Farmer  Skinner  raises  pigeons 

FARMERS  BITS  OF  INFORMATION  SHEET  (F) 

Only  Skinner  lives  at  the  west  end  of  the  village 
There  are  albino  rats  in  the  yard  of  the  ranch  house 
One  of  the  groups  tasks  is  to  decide  who  grows  the  apples 
Pavlov  lives  in  the  log  cabin 

Each  farmer  lives  in  a  different  typ^of  house 
Thorndike  lives  between  the  bungalow  and  log  cabin 
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Answer : 


Skinner  drives  a  truck 


Hull  raises  apples 

Anything  else  in  the  solution  is  incorrect. 

Break 

Ivaluatjon 

Review  Objectives 

*****Review  the  objectives  at  the  beginning  of 

Importance  of  Individual js 
Contribution 

How  was  this  exercise  similar  to  team  decision 
might  encounter  on  the  job? 

#*»#*Discuss  this  question. 


jO  min 

2  min 

this  session 

5  min 

making  you 


#####Note  that  in  the  excercise  each  individual  had 

essential  information.  What  if  one  person’s  input  had 
been  left  out? 

*####Ask  whose  responsibility  it  is  to  see  that  everyone 
gives  input. 
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Understanding  and  Identifying 

Roles  28  min 

#####Go  through  the  list  of  roles  and  discuss  who  in  each 
group  was  observed  to  behave  similarly  to  that  role. 
List  specific  things  they  did. 

Ro  1.  e  Who!  How? 

Initiator 

Seeker 


Giver 


Summar izer 


Harmonizer 


Gate  Keeper 


Encourager 


Compromiser 


Standard  Tester 
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How? 


Role  Who? 

Aggressor 


Blocker 


Dominator 


Playboy 


Faci 1 1 tators 


lBi£°duct^on  §  min 

What  is  a  facilitator? 

»««»*Define  a  facilitator  as  an  asset  to  the  group  assigned 
responsibility  for  group  dynamics,  and  for  keeping  the 
gr^xjp  on  track  and  functioning  effectively. 

tt»***Ask  the  following  question. 

What  actions  might  a  facilitator  take  in  a  group  decision 
making  session? 
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Response  to  Roles  1^0  min 

How  might  a  facilitator  respond  to  each  of  these  roles? 

*####Discuss  how  a  facilitator  might  respond  to  these  roles 
people  adopt  in  a  group  setting.  Encourage  the  group 
to  look  back  at  the  individual  members  of  their  team  in 
the  excercise,  and  think  how  a  facilitator  might 
respond  to  improve  the  group  effectiveness. 

Initiator 


Seeker 


Giver 


Summar izer 


Harmon izer 


Gate  Keeper 


Encourager 


Compromiser 


Standard  Tester 


Aggressor 


Blocker 


Dominator 


Playboy 
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Session  7  Training  Objectives  and  Pitfalls 

Objectives:  Establish  the  broad  training  goal  of  creating 

the  cultural  change  needed  for  a  successful 
program. 

Define  the  more  specific  training  goals  of 
imparting  tools  for  use  in  a  performance 
management  framework. 

Introduce  some  of  the  most  common  problems 
and  errors  encountered  in  training  programs. 


Lesson  P_1  an Introduction  .10  min 

***** Ask  what  kinds  of  training  individuals  have  been 

through  before  in  association  with  their  jobs.  List 
some  . 

*****Ask  how  useful  the  training  was.  In  what  ways  was  it 
useful?  What  parts  were  not  worth  the  effort? 

*****Ask  what  are  some  elements  training  should  include. 

Where  should  the  line  be  drawn  where  further  training 
is  not  worth  the  effort? 

*****Review  the  session  objectives  above. 

Cultural  Change  Training  JL 5  mi  D 

What  is  cultural  change? 

*****Note  that  this  subject  was  mentioned  in  the  morning’s 
lessons  about  management’s  role.  Ask  the  group  to 
review  the  meaning  of  cultural  change.  Ask  how  it  is 
accomplished.  Ask  why  is  it  necessary. 
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How  can  training  be  used  to  begin  this  change? 

*##*»Ask  if  a  Man  from  Mars  exercise  is  valuable?  Why  or 
why  not? 

***** Ask  if  having  the  informal  leaders  give  testimonials 
would  be  effective.  Why  or  why  not? 

#####Would  having  the  squadron  commander  come  and  challenge 
the  trainees  work?  Why  or  why  not? 

#####Would  having  someone  whom  everyone  would  think  to  be 

the  last  to  support  such  an  improvement  effort,  in  fact 
to  resist  it,  come  and  give  support  be  a  help?  Why  or 
why  not? 

##*#*Ask  for  other  suggestions. 

#*#*#Ask  where  this  training  should  be  accomplished,  and  by 
whom. 


Where  should  this  training  be  accomplished? 


Specific  Tools  Training  10  min 

*#*»*Comment  that  in  addition  to  training  for  cultural 
change  objectives,  there  are  particular  tools  and 
specific  techniques  to  be  understood  by  various 
individuals  involved  in  a  performance  management 
framework,  such  as  measurement  methods,  evaluation 
techniques.  Ask: 

How  might  training  for  use  of  specific  performance 
management  framework  tools  differ  from  the  cultural  change 
training? 

##*«#Ask  what  the  objectives  of  this  training  would  be.  How 
detailed  does  this  training  need  to  be  compared  to  the 
cultural  change  training. 


*  #  # * » Ask : 

Who  would  you  recommend  to  lead  this  training?  Where  might 
this  training  be  accomplished? 
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Problems 


15  min 


*#**#Inform  the  group  that  now  we  will  turn  our  attention  to 
some  common  problems  that  arise  in  training  programs. 
The  first  is  going  'training  happy"  and  steamrolling 
everyone  through  training  right  away.  Ask: 

What  problems  might  arise  from  training  everyone  right  away? 

#»*##Do  you  see  any  advantages  to  such  a  plan.  Do  you  think 
the  benefits  outweigh  the  costs.  If  they  seem  stuck, 
ask  them  to  think  about  training  they  have  been 
through,  and  if  any  conclusions  can  be  drawn. 

»**##Note  that  a  related  symptom  is  seeing  the  interest 

become  nothing  more  than  a  slogan  program.  People  need 
to  be  able  to  see  real  changes  and  commitment  along 
with  the  talk. 


What  problems  do  you  foresee  from  training  people  too 
extensively,  or  in  too  much  detail  for  their  needs? 

##«#*Desor ibe  another  problem  as  the  shotgun  approach  to 

training.  Give  them  everything,  whether  they  need  it 
or  not.  Ask  what  harm  they  see  in  this.  Is  it 
possible  to  lose  sight  of  the  forest  for  the  trees? 

**#**Ask  if  the  group  sees  any  advantages  in  this  approach. 
Do  they  think  the  benefits  outweigh  the  costs? 


What  problems  would  be  associated  with  having  no  training? 

###*#Comment  that  another  problem  is  just  the  opposite  of 
what  we  have  been  discussing,  and  that  is  the  sink  or 
swim  method.  No  training  is  given.  Ask  what  harm  the 
group  can  see  in  this. 

»###*Ask  for  any  benefits  the  group  can  think  of.  Ask  if 
the  benefits  outweigh  the  costs. 


*»***Summarize  and  wrap  up. 
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Session  8  Training  Techniques  and  Strategies 

Objectives:  Identify  which  individuals  in  the  Engineering 

Branch  should  receive  training.  Suggest  what 
factors  are  pertinent  to  choosing  these 
people . 

Determine  the  most  opportune  time  to  provide 
various  degrees  of  training.  List  important 
criteria  in  selecting  this  correct  time  in 
different  situations. 

List  the  essential  ingredients  to  be  included 
in  the  training  curriculum.  What  situational 
factors  might  require  revising  these 
ingredients? 


t®§son  Introduction  3  min 

#####Remind  the  group  that  last  hour  they  looked  at  some  of 
the  objectives  and  problems  of  training.  Inform  them 
that  this  hour  they  will  have  an  opportunity  to  look  in 
more  detail  at  overcoming  these  problems  by  examining 
who,  what,  and  when  to  train. 

»»»«*Review  the  lesson  objectives  above. 

Team  Project  32  min 

«»*»«Divide  the  group  into  three  teams.  Give  a  list  of 

questions  to  be  discussed  to  a  representative  from  each 
team.  Instruct  them  to  generate  a  training  plan  or 
answers  to  the  questions  given.  Note  that  each  team 
will  have  a  five  minute  opportunity  to  brief  their 
conclusions  to  the  group.  Tell  them  they  have  30 
minutes  to  prepare . 

#*»**Fol lowing  are  the  lists  of  questions  for  each  team. 
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Team  1:  Topic  -  Who  to  train? 

Assignment  -  Develop  a  list  of  whom  within  the 

organization  you  would  train.  What  type  of 
training  (cultural  change,  specific  tools, 
other)  should  they  receive?  It  is  not 
necessary  to  detail  the  content  of  the 
training,  just  identify  the  overall  theme 
proposed . 

Questions  to  consider  - 

What  are  the  criteria  you  use  to  select  the 
people  for  each  type  of  training? 


Where  there  are  several  individuals  within 
the  same  work  area  having  the  same  job 
description,  what  factors  affect  who  you  pick 
for  training?  Or  do  you  train  them  all 
alike?  Why? 


Who  would  you  have  provide  the  training? 
Why? 


What  specific  objectives  would  you  set  for 
each  type  of  training? 
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When  to  train 


Team  2:  Topic  - 

Assignment  -  Develop  guidelines  for  when  you  would  provide 
training  within  your  organization  for 
differing  types  of  training  (cultural  change, 
specific  tools,  other). 

Questions  to  consider  - 

Do  you  want  to  have  all  the  training  an 
individual  receives  provided  at  one  time,  or 
give  different  levels  of  training  at 
different  times?  Why? 


What  kind  of  schedule  of  getting  people  each 
type  of  training  would  you  follow? 


What  situational  factors  would  affect  your 
training  schedule  guidelines?  What  are  your 
criteria  for  deciding  when  to  train? 


How  often,  if  ever,  would  you  have  refresher 
courses  ? 


What  objectives  would  you  set  for  each  type 
of  training? 
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Team  3:  Topic  -  What  to  train 

Assignment  -  Develop  the  components  that  you  would  put 

into  the  contents  of  each  type  (cultural 
change,  specific  tools,  other)  of  training 
course  ? 

Questions  to  consider  - 

What  things  are  essential  for  the  trainees  to 
come  away  with? 


Should  the  content  be  the  same  for  everyone1? 
Why  or  why  not? 


What  important  ingredients  would  need  to  be 
included  in  the  training? 


What  objectives  would  you  set  for  each  type 
of  training? 
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Presentations 


15  min 


### * #Prov ide  each  team  five  minutes  to  present  their  plan 
and  answer  questions. 
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Session  9 
Objectives : 


Reference : 


Process  Improvement 

Present  the  concepts  associated  with  viewing 
organizational  activity  from  a  process 
perspective . 

Understand  the  five  principal  features  of 
process  improvement  plans. 

Define  cross-f unctional  processes.  Establish 
how  to  determine  the  input  and  output  which 
flow  between  these  producers  and  consumers. 

Discuss  ownership  of  processes.  Show  the 
importance  of  assigning  responsibility  for  a 
process  to  a  single  individual. 

Communicate  the  necessity  of  extending  the 
process  to  its  origin  and  conclusion.  The 
greatest  improvements  for  the  least  expense 
can  often  be  made  by  thorough  planning  at  the 
start  of  the  process. 

Process  improvement  plans  -  Total  Quality 
Assurance  Through  Process  Management .  by 
Kenneth  R.  Jennings  and  others,  1989,  12-89. 


Lesson  Plan^  iDiloduction  3  min 

#####Review  the  lesson  objectives  above. 

Process  Perspective  7  nun 

Definition 

»####Define  process  as  all  activities  which  are 

performed  to  accomplish  a  recurring  organizational 
ob j  ective . 

Repeatable 

#####Note  that  to  qualify  as  a  process  the  procedure 
must  be  repeatable,  something  that  the 
organization  does  many  times. 
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Universal 


#»##*State  that  all  workplace  activities  fall  into  some 
process.  Ask  the  group  if  they  can  think  of  any 
things  they  do  which  do  not  fall  within  a  process. 
If  they  suggest  work  on  individual  unique 
projects,  reply  that  even  though  the  project  is 
unlike  others,  the  procedure  followed  to  define 
the  project  and  solve  it  is  a  repeatable  process. 

Relate  to  big  picture 

#####Note  that  viewing  the  individual  tasks  each  worker 
does  as  a  part  of  a  larger  process  helps  create 
the  process  perspective. 

Building  blocks  of  performance 

#####Ask  the  group  to  what  degree  organizational 

performance  depends  on  how  well  each  of  their 
processes  are  accomplished?  If  they  do  well  on 
each  of  the  processes,  will  they  necessarily  do 
well  overall?  Why  or  why  not? 

Key  to  improvement 

###»«Ask  the  group  to  describe  the  usefulness  of  a 
process  perspective  on  the  work  they  do  with 
regard  to  identifying  opportunities  for 
improvement . 

»»##*Note  that  many  opportunities  for  improvement  exist 
at  the  communication  interfaces  between  steps  in 
these  processes. 
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Process  Improvement  Plans 


15  min 


Def inition 

#####Explain  that  the  first  step  in  process  improvement 
is  understanding  the  structure  and  steps  of  the 
process.  Explain  the  diagram  below  by  pointing 
out  the  the  process  begins  at  the  point  some 
external  supplier  provides  input.  It  continues 
until  some  output  is  provided  to  a  user.  Along 
the  way  are  all  the  individual  steps  of  the 
process  which  must  be  accomplished  to  transform 
the  input  to  output. 

*####Ask  the  group  if  this  kind  of  model  would  apply  to 
the  processes  that  make  up  their  work.  Do  they 
have  suppliers  and  users  or  customers?  Why  or  why 
not? 


Suppliers  Activity  Steps  Users 

Input  - >  Transformation  - >  Output 


*##**Ask  the  group  how  they  would  go  about  identifying 
the  activity  steps  that  they  perform  in  this 
process . 


Simpl i f ication 

simplify  the  process,  see  if  any  of  the 
activity  steps  are  repetitive  or  can  be 
eliminated.  Ask  the  group  how  this  step  alone 
might  improve  their  work  processes.  What  would 
the  results  be? 

Measurement 

##*#*Explain  that  specifications  are  used  for 

measurement.  The  input  and  output  crossing 
boundaries  between  processes  or  steps  within  the 
same  process  are  compared  to  a  standard  or  average 
specification.  Ask  the  group  to  describe  how 
these  variations  from  specifications  might  be 
used  to  better  understand  the  process  and  the 
opportunities  for  improvement  within  the  process. 

*»«»»What  kinds  of  things  might  be  measured  in  the 
organization’s  work  processes?  Point  out  that 
these  are  the  factors  which  can  be  improved. 
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Control 


##«##Explain  that  the  control  step  is  to  determine 

where  and  what  controls  the  factors  measured  in 
each  step  of  the  process.  How  might  these  control 
points  be  found?  What  tools  could  be  used  to  aid 
this  task? 


Improvement 

Identify  Opportunities 

»####Note  that  once  the  process  is  defined  and 

simplified,  and  the  variable  factors  within 
it  are  measured  and  their  points  of  control 
identified,  the  opportunities  for  improvement 
can  be  determined.  List  these  opportunities 
for  each  step  of  the  process. 


Cause  and  Effect  Planning 

*****Remark  that  a  useful  tool  in  establishing  the 
causes  and  potential  solutions  for  problems 
is  the  cause  and  effect  technique.  The  use 
of  this  tool  will  be  discussed  in  more  detail 
later.  Its  use  results  in  discovering  the 
most  likely  causes  for  problems  in  the 
process . 

Action 

#####Once  the  caus  •  for  problems  have  been  listed 
for  the  improvement  opportunity,  actions  to 
implement  a  solution  need  to  be  arrived  at. 
Ask  the  group  to  list  important  things  to 
consider  when  proposing  an  action? 

#**#*If  these  questions  are  not  answered  in  the 
discussion,  bring  them  up:  How  is  the  best 
of  the  solution  alternatives  selected?  Who 
is  responsible  to  see  the  action  through? 

How  should  this  responsibility  be  delegated? 
How  might  essential  resources  be  secured? 
Should  a  deadline  be  set?  Who  can  approve 
the  action? 
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Cross-functional  Processes 


10  min 


*»ft»«Note  that  the  subject  of  multi-functional  issues  in 

organizations  has  been  discussed  before  in  this  course. 

Now  we  will  look  specifically  at  the  issue  of  multi-  or 

cross-functional  processes. 

Functional  interdependence 

»ft»*«Ask  the  group  to  describe  considerations  which 

arise  when  one  independent  functional  group  within 
an  organization  depends  on  another  in  its  work. 

Input 

«*tt«*Define  input  in  this  context  as  the  product  or 
information  that  one  function  requires  for  its 
work.  Ask  the  group  to  discuss  difficulties  that 
functional  areas  sometimes  have  in  obtaining  the 
input  they  need. 

Output 

##*##Define  output  as  the  product  or  information  that 
one  function  issues  as  a  result  of  its  work.  Ask 
the  group  to  list  some  of  the  impacts  the  function 
can  have  on  other  functions  through  its  delivery 
of  this  output. 

Coordination 


***««Comment  that  you  think  the  group  will  agree  that 
coordination  between  functions  with  input-output 
relationships  is  essential  to  the  performance  of 
the  organization.  Ask  the  group  to  suggest 
reasons  why  this  coordination  is  often  so 
difficult  and  in  many  cases  does  not  get  done. 

»»ft»«If  the  group  has  not  suggested  this  after  some 
discussion,  note  that  often  this  coordination 
between  functions  is  lost  because  the  functions 
are  not  in  the  same  chain  of  command. 

*  *  *  *  * Ask  the  group  what  the  result  of  breakdown  in 
coordination  of  this  type  might  be. 

»ft»ftftAsk  if  this  means  no  one  is  responsible  for  seeing 
that  the  process  proceeds.  Note  that  the  fact  of 
life  is  that  someone  is  always  responsible. 
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Process  Ownership 


5  min 


Responsibi 1 ity 

#####Ask  the  group  who  it  is  who  is  usually  held 
responsible . 

#####Comment  that  the  person  most  knowledgable  about 
the  process ,  or  who  controls  the  largest  portion 
of  the  process,  will  naturally  be  the  person  to 
make  responsible  for  the  process.  This  person  is 
called  the  process  'owner.'  (If  discussion 
develops  that  someone  in  a  higher  supervisory  role 
is  responsible,  Just  note  that  carried  to  the 
extreme  the  organization  chief  is  ultimately 
responsible  for  all  the  processes,  but  that 
responsibility  is  delegated  to  the  person  who 
actually  does  the  work  as  being  most 
knowledgabl e . ) 

Authority 

###*#Note  that  the  owner  will  often  not  have  authority 
over  some  of  the  other  functional  areas  which  work 
on  the  process.  Ask  the  group  to  discuss  problems 
this  presents. 


Pride 

#*«#*Ask  the  group  how  'pride  of  ownership'  can 

contribute  to  owners  efforts  to  see  that  the 
processes  for  which  they  are  responsible  run 
smoothly,  and  are  improved. 
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Process  Extension 


10  min 


Downstream 

****»Downstream  extension  is  pushing  the  process 

description  to  include  steps  further  toward  the 
end  user  or  conclusion  of  the  process.  Ask  the 
group  to  suggest  why  this  might  increase 
understanding  of  the  process.  How  would  this 
assist  in  determining  the  impacts  of  steps  taken 
earlier  in  the  process? 

Upstream 

###*#Define  upstream  extension  as  pushing  the  process 
definition  to  include  steps  further  toward  the 
conception  and  development  of  the  needs  which 
require  the  process.  Ask  the  group  to  suggest  why 
this  might  offer  potential  for  improvements  in 
organizational  performance. 

**###Note  that  often  the  greatest  improvements  for  the 
least  cost  can  be  made  at  early  stages  in  the 
process.  Relatively  few  resources  have  been 
committed  and  small  improvements  will  be 
multiplied  many  times  over  when  carried  through 
the  whole  process.  Ask  the  group  who  it  is  who 
makes  the  decisions  regarding  conception  and 
development  of  needs  at  these  early  stages  in  the 
process.  Ask  the  group  how  these  decision  makers 
might  make  better  decisions  in  the  long  run. 

Di f  f iculties 

#####Ask  the  group  if  it  seems  futile  to  extend  the 
^roc-css  up* bream  to  consider  these  critical 
developmental  decisions,  when  often  these 
decisions  are  out  of  control  of  the  process  owner. 

###**Note  that  often  it  is  difficult  because  no  one 

really  knows  how  these  decisions  are  made,  and  it 
takes  a  lot  of  digging  to  find  out.  Ask  the  group 
if  this  digging  would  be  worthwhile.  Why  or  why 
not? 
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Session  10  Process  Flow  Exercise 

Objectives:  To  allow  the  group  to  identify  a  process  in  a 

multi-functional  environment.  The  process 
interfaces  will  be  determined,  along  with  the 
input  and  output  passed  between  these 
functions . 


Lesson  Plan;  Introduction  3  min 

##*##Review  that  last  hour  we  looked  at  the  process 

perspective,  how  to  view  work  activities  as  they  fit 
into  the  bigger  picture  of  processes. 

#»###This  hour  we  will  take  a  familiar  example  from  everyday 
experience  and  see  how  to  detail  the  process. 

*####Review  the  lesson  objectives  above. 

Process  Excercise 


iDiESduct ion  2  min 

####* Announce  that  the  process  to  be  considered  is  getting 
your  body  out  of  bed  in  the  morning  and  getting  it  to 
work.  Elaborate  that  this  is  a  typical  day,  there  is 
work  to  be  done,  no  sneaking  off  to  the  golf  course  or 
the  tennis  courts,  it’s  out  of  bed  and  to  work. 


Enochs s  Flow  Diagraming  i§  min 

#####Have  the  group  list  the  steps  in  getting  to  work. 
Write  them  down  with  plenty  of  room  to  fit  steps 
suggested  later  in  between  earlier  ones.  Take  your 
time.  Qet  all  the  details.  Show  some  enthusiasm  as 
needed  to  get  the  group  rolling. 


Cross;! unot ional  aspects 

Function  Identification 


4 

6  min 


##**#Ask  the  group  to  identify  functional  areas  within  this 
process.  Note  that  even  though  for  the  most  part  all 
operations  are  performed  by  yourself,  consider  yourself 
to  be  performing  different  functions  as  makes  sense. 


##*#»Have  the  group  describe  the  interfaces  between  these 
functional  areas.  What  is  the  input-output  passing 
from  one  to  another? 
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Input  Identification 


6  min 

»»#*«Ask  the  group  to  consider  the  process  they  have 

identified  and  note  where  input  is  needed  from  outside 
the  process.  What  is  this  input  and  who  provides  it? 

Output  Identification  6  min 

***»*Have  the  group  consider  the  process  they  have 

identified  and  note  where  output  is  issued  out  of  the 
process.  What  is  this  output?  Who  is  it  provided  to? 
How  does  it  impact  the  process? 

Ownership  2  min 

***#*Ask  who  is  the  owner  of  this  process?  What  does  this 
mean? 


Extension  _1 0  min 

#»##*Ask  the  group  to  extend  the  process  even  further  up  and 
downstream,  if  possible.  Ask  what  decisions,  factors, 
or  occurances  impact  the  process  from  the  lead  or 
upstream  end. 

#«*##Ask  the  group  to  describe  the  effect  on  overall 

performance  that  a  small  change  made  at  the  upstream 
end  could  produce. 


Session  11 
Objectives : 


Materials : 


Process  Flow-charting 

To  provide  further  experience  in  describing 
the  underlying  processes  comprising  the 
workplace . 

Standard  job  descriptions  or  occupational 
instructions  for  a  programmer,  a  design 
engineer,  and  an  construction  inspector  at 
your  base. 


Lesson  PI an^  5  min 

**###Tell  the  group  that  this  hour  they  will  have  an 

opportunity  to  apply  the  process  identification  tools 
we  have  learned  to  situations  they  are  familiar  with, 
but  are  a  bit  more  realistic  for  a  work  situation. 

####«Review  the  lesson  objectives  above. 

#####Assign  and  distribute  the  three  job  descriptions  as 

cases,  one  to  each  person,  so  there  are  an  equivalent 
number  of  people  for  each  case.  Explain  they  should 
read  the  case  and  pay  attention  to  the  processes  they 
can  identify.  They  will  have  a  chance  to  discuss  the 
details  shortly.  Tell  them  they  have  five  minutes  to 
acquaint  themselves  with  their  cases. 


Read  Cases  5  min 

Panel  Discussion  36  min 

#»*#»Have  a  team  of  members  who  all  read  the  same  case 

assemble  at  the  front  of  the  room.  Inform  them  that 
this  is  a  panel  discussion  type  forum.  They  are  to 
respond  to  the  questions  given.  Interaction  and 
alternate  opinions  by  other  panel  members  are 
encouraged.  Ask  for  a  volunteer  from  the  panel  to 
volunteer  to  summarize  the  case.  Move  through  the 
questions  as  the  panel  is  ready,  or  as  time  requires. 
Each  panel  has  twelve  minutes.  Suggested  times  for 
each  question  are  given.  When  one  team  is  done, 
repeat  the  panel  procedure  with  each  of  the  other 
teams . 
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Summarize  the  case. 


2  min 


Describe  the  processes  evident  in  the  case.  List  the  steps 
which  comprise  these  processes. 

3  min 


Which  separate  functional  units  are  interfaced  with  in  each 
process  ? 

2  min 


Who  is  the  process  owner?  Over  which  steps  in  the  process 
does  the  owner  have  personal  authority? 

2  min 


What  are  elements  of  input  and  output  evident  in  these 
processes? 

3  min 


Q2nslysi2n  1  miQ 

###*»Ask  and  discuss: 

How  can  defining  the  process  in  work  situations  assist  in 
managing  and  improving  the  organization's  performance? 

»#»##Tell  the  group  that  the  rest  of  the  afternoon  we  will 
look  at  the  definition  of  performance  and  how  to 
measure  various  aspects  of  performance. 
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Session  12  Definition  of  Performance  &  Customer 

Satisfaction 

Objectives:  List  and  define  the  seven  components  of 

performance.  Discuss  which  are  most 
appropriate  for  the  Engineering  Branch. 

Understand  the  important  role  of  customer  or 
user  satisfaction  in  improving  performance. 
Identify  who  customers  are  of  the  Engineering 
Branch . 

Reference:  Planning  and  Measurement  in  Your  Organization 

of  the  Future,  by  D.  Scott  Sink  and  Thomas 
C.  Tuttle,  1989,  ppl71-186. 


Lesson  P 1  anj.  Introduction  5  min 

*»»*#Review  the  lesson  objectives  above. 

#####List  and  briefly  define  the  seven  components  of 

performance.  Tell  the  group  these  are  the  components 
of  performance  we  will  be  discussing. 

Effectiveness  -  How  well  you  get  the  job  done 
Efficiency  -  How  well  you  utilize  your  resources 
Productivity  -  The  ratio  of  output  to  input 
Quality  -  The  degree  to  which  your  processes  are 
accomplished  well 

Innovation  -  Your  creative  improvement  ideas 
Quality  of  Work  Life  -  The  attractiveness  of  the 
workplace 

Budgetabi 1 i ty  -  How  well  you  meet  your  resource 
constraints 

E®£i2£!3?§D2®  Component  Definition 

##«##For  each  con^ponent,  give  the  operational  definition. 

##*##Ask  the  group  for  applications  within  their 

organization.  When  several  have  been  suggested,  ask 
the  group  if  they  agree  that  each  is  a  correct 
application  of  the  component.  Why  or  why  not? 

##*##Ask  in  what  ways  the  group  feels  that  component  is 
important  in  defining  their  performance. 

**#*#Have  the  group  members  refer  to  the  performance 

component  diagram  at  the  end  of  the  lesson  as  needed 
with  each  component  to  assure  an  understanding  of  the 
meanings . 
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Performance  Components 

Ef f ecti veness  4  min 

Operational  Definition  - 

»###*The  ratio  of  actual  output  to  expected 

output.  This  is  an  output  side  component. 

Possible  applications  - 


How  important  is  this  as  a  component  of 
per  f  ormance? 


Efficiency  4  min 

Operational  Definition  - 

#»*«*The  ratio  of  expected  input  to  actual  input. 
rihis  is  an  input  side  component. 

Possible  applications  - 


How  important  is  this  as  a  component  of 
per  f  ormanor ? 


4 
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Productivity 


4 

Operational  Definition  - 

*####The  ratio  of  output  to  input  in  the 
units. 

Possible  applications  - 


How  important  is  this  as  a  component  of 
performance'5 


Quality  4 

Operational  Definition  - 

#**##The  degree  to  which  each  step  of  the 
and  the  process  as  a  whole,  operates 
correctly . 

Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 


4 


min 


same 


min 


process  , 
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Innovat i on 


4  min 


Operational  Definition  - 

#**##An  index  of  the  creative  process  of  changing 
to  respond  to  new  pressures,  opportunities, 
and  threats . 

Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 


Quality  of  Work  Life  4  min 

Operational  Definition  - 

*#»##The  reaction  of  the  organization’s  personnel 
to  the  work  environment,  including  factors 
such  as  pay,  leadership,  autonomy, 
involvement,  and  relationships. 

Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 


4 
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Budgetabi 1  i  ty 


4  min 


Operational  Definition  - 

###*#The  relationship  of  budgets  and  goals  with 
actual  costs  and  accomplishments.  This 
component  is  comparable  to  profitability  in  a 
private  organization. 

Possible  applications  - 


How  important  is  this  as  a  component  of 
performance? 


***»*Note  that  there  is  a  diagram  at  the  end  of  the  lesson 
showing  the  process  model  and  its  relation  to  these 
seven  performance  components.  Have  the  students  flip 
to  the  diagram  and  ask  if  there  are  any  questions. 

Component  Importance  2  min 


*  *  *  »  » Ask : 

Effectiveness  and  quality  have  been  suggested  as  the  two 
components  most  critical  to  the  performance  of  a  civil 
engineering  organization.  The  other  components  depend  on 
these  two.  Do  you  agree’  Why  might  you  think  this  to  be 
the  case1? 


Customer  Satisfaction 

####*Note  that  several  of  the  above  components  of 

performance  are  concerned  with  the  user’s  attitude 
toward  the  organization's  output.  To  borrow  a  term  from 
the  commercial  world,  customer  satisfaction  is  an 
important  perspective  for  any  organization. 
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Importance  of  Customers 


5  min 


*****Ask : 

Why  is  customer  satisfaction  important  to  the  Engineering 
Branch? 

###»#Why  should  you  care  what  they  think? 

*##*#If  you  have  designed  a  good  air  conditioning  system  or 
parking  lot,  what  concern  should  you  have  with  the 
user’s  satisfaction? 


Identification  of  Customers  1.0  min 


#***»Ask : 

Who  are  customers  of  the  Engineering  Branch? 

*  * *##Elaborate  that  we  are  not  interested  just  in  other 

organizations  C.E.  builds  and  maintains  facilities  for. 
Who  are  the  organizations  that  use  the  products  the 
Engineering  Branch  produces? 

##*#*If  base  contracting  and  contractors  themselves  have  not 
been  mentioned  after  some  discussion,  ask:  Can  you 
consider  contracting  a  customer?  Can  you  consider  the 
contractors  who  bid  and  build  your  projects  as 
customers? 


#####Ask : 

How  does  the  Engineering  Branch  serve  each  of  these 
customers?  How  can  they  be  better  served? 
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PERFORMANCE  COMPONENT  DIAGRAM 


USER  -- 
NEED 


Note  : 


- Budgetabi I i ty 

Productivity - 


Innovation 


--  INPUT 


TRANSFORMATION 


Quality  of 
Work  Life 


OUTPUT USER 

!  SATISFACTION 


Ef  f iciency 


Ef  f ecti veness 


Qual i ty 


The  definitions  and  diagram  for  the  performance 
components  presented  in  this  lesson  are  based  on 
those  proposed  by  D.  Scott  Sink  and  Thomas  C. 
Tuttle  in  their  book  Planning  and  Measurement  in 
Yayi!  Ocgsoizaiiaa  si  ibs  Eyiycs ,  Norcross , 
Georgia:  Industrial  Engineering  and  Management 
Press ,  1989 . 
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Session  13 


Application  of  Performance  Components 


Objectives:  Apply  the  components  of  performance  to  the 

Engineering  Branch  environment.  Suggest 
elements  of  performance  which  could  be 
targeted  for  evaluation  and  improvement  under 
a  performance  management  framework. 


I_n tr oduc t i^on  5  min 

»#*#»Recap  the  performance  components  from  last  lesson. 
Remind  the  group  that  customer  satisfaction  is  an 
important  perspective  for  engineers. 

***»#Review  the  lesson  objectives  above. 

Paired  application  analysis  25  min 

#####Divide  the  group  into  pairs,  making  a  team  of  three  if 
there  is  an  odd  number.  Remind  the  group  that  in  last 
hour’s  discussion  potential  applications  for  each  of 
the  components  of  performance  were  briefly  discussed. 
Now  your  task  will  be  to  consider  Just  one  of  these  in 
more  detail.  Assign  the  components  to  teams,  one  per 
team,  in  the  following  order:  effectiveness,  quality, 

efficiency,  producti vi ty ,  budgetabi 1 i ty ,  innovation, 
and  quality  of  work  life.  Tell  them  their  assignment 
is  in  their  handbook.  They  have  25  minutes.  Then  each 
pair  will  present  their  conclusions  to  the  group. 

Critically  analyze  one  of  the  components  of  performance  in 
its  application  in  your  organization. 

k  4<*t  is  a  reasonable  operational  definition  of  this  aspect 
of  performance?  How  can  it  be  measured  or  evaluated  in  your 
organization? 

Suggest  how  this  would  allow  managers  to  track  performance 
4  and  determine  where  improvement  is  possible.  How  would  the 

improvement  be  recognized  and  reported? 

20  min 

*****Have  each  pair  present  their  conclusions,  3-5  minutes 
per  pair. 
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Session  14 


Implementation  Details 


Objectives:  Outline  the  steps  involved  in  a  top-down 

implementation  of  a  Performance  Management 
Framework . 

Illustrate  the  technique  of  using  a  pilot 
project  to  initiate  the  program. 

Describe  the  importance  of  the  facilitator  to 
the  operation  of  the  team  function. 

Lesson  PI an^_  Introduction  3  min 

###*#Review  the  objectives. 

Announce  that  now  that  we  have  spent  the  last  two  days 
discussing  the  tools  and  outlook  needed  to  use  the 
Performance  Management  Framework,  it  is  time  to  pull 
these  ideas  together  and  see  how  they  are  used  to 
improve  the  work  in  an  organization. 

Paired  individual  analysis  25  min 

###**Divide  the  group  into  teams  of  two.  Assign  one  of  the 
five  main  topics  to  each  group.  Have  them  consider  the 
individual  issues  under  each  topic,  and  prepare  to 
brief  the  class  according  to  the  following  instruction. 
Tell  them  they  have  20  -  25  minutes  to  prepare  their 
presentations,  and  then  each  team  will  have  3-5 
minutes  to  present.  If  there  are  insufficient  teams  to 
cover  all  five  topics,  combine  first  Management 
direction  with  Management  support,  second  Team  process 
definition  with  Team  improvement  efforts. 

Describe  how  the  following  need  to  be  addressed  within  the 
context  of  the  Performance  Management  Framework  in  order  to 
promote  successful  improvement  efforts. 

Management  direction 

Strategic  planning 

Identification  of  principal  processes  comprising  work 

Selection  of  process  as  pilot  project  for  first 
improvement  effort 

Selection  of  process  for  next  team  improvement  effort 
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Management  support 

Assignment  of  members  to  the  team 
Training  decisions 

Selection  of  a  facilitator  to  assist  team 
Team  process  definition 
Steps 

Input  and  output 

Ownership 

Interfaces 

Team  improvement  efforts 
Identify  problems 
Propose  solutions 
Recommend  actions 

Who  responsible 
Schedule  and  milestones 
How  to  measure  for  success 
Management  implementation 

Screen  implementation  actions 

Support  implementation  actions  selected 

Reward  efforts 

i  ons  25  D?iO 

###*#Have  teams  take  turns  presenting  their  conclusions, 
allowing  3-5  minutes  per  team. 

2  min 

*»»**Inform  the  group  that  they  should  now  have  a  complete 
picture  of  how  to  go  about  conducting  the  Performance 
Management  Framework  in  an  organization.  Announce  that 
for  the  rest  of  the  morning  they  will  have  a  chance  to 
go  through  these  steps  in  an  exercise  environment. 
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Session  15 
Ob j  ectives : 


Materials : 


Implementation  Exercise 

Provide  a  situation  in  which  members  can 
experience  implementation  of  the  Performance 
Management  Framework. 

Discuss  the  lessons  learned  and  how  they 
might  be  applied  to  the  program 
implementation  in  the  Engineering  Branch 
Environment . 

Several  dozen  8-1/2  x  11  sheets  card  stock  or 
other  heavy  construction  paper  (buff  color  is 
fine),  masking  tape,  unsharpened  pencils, 
typing  paper,  sciss<-j3,  12  inch  scales, 
drafting  triangles. 


Lesson  Plan Introduction  5  min 

*#*#*Tell  the  group  that  the  rest  of  the  morning  will  be  a 
workshop  exercise  to  practice  the  principles  of  the 
Performance  Management  Framework.  They  will  have  a 
chance  to  look  at  a  situation  from  both  a  management 
and  worker  perspective. 

Exercise 

Situation  30  min 

*««*xRead  the  following  description  of  the  situation. 

You  are  an  engineering  branch  comprised  of  three  sections, 
programming,  design,  and  construction  management.  Due  to 
the  limited  scope  of  this  exercise,  construction  management 
will  actually  do  the  construction  called  for  in  the 
exercise.  Your  task  is  to  program,  design  and  build  the 
items  identified. 
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Part  of  this  task  requires  costing  of  labor  and  materials  to 
be  used.  Costs  are: 


Line  Item 

Cost 

Whole  sheets  of  construction  paper 

*10  / 

sheet 

Unsharpened  pencils 

*  5  / 

each 

Tape 

*  1  / 

inch 

Labor  by  builder 

*  2  / 

minute 

Note  that  sheets  of  paper  are  charged  as  whole  sheets,  even 
if  only  a  portion  is  used.  The  left  over  cannot  be  used  on 
another  project.  Tape  cannot  be  used  in  shorter  than  one 
inch  pieces.  Pencils  cannot  be  broken. 

#***#Divide  the  group  into  teams  of  3  -  6  people.  Within 

each  team,  1  or  2  people  will  become  programmers,  1  or 
2  will  be  designers,  and  1  or  2  will  be  construction 
management.  Each  set  will  perform  the  same  exercise 
below.  Inform  all  members  that  one  project  is  design 
complete  and  has  been  on  the  shelf,  but  Just  got 
funded.  Give  a  copy  of  this  to  all  the  construction 
management  people.  One  project  is  programmed,  awaiting 
design.  Give  a  copy  of  this  to  the  designers.  There 
is  a  pile  of  projects  waiting  to  be  programmed.  Give  a 
copy  of  these  to  the  programmers. 

###*#Give  a  copy  of  the  following  written  instructions  to 

the  programmers,  designers,  and  construction  managers. 
Only  give  the  appropriate  instructions  to  each  member. 
For  instance,  the  designers  would  get  the  designer 
instructions  but  not  the  programmer  or  construction 
manager  instructions. 

»#*#*Ask  for  questions.  Tell  them  to  begin. 
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####»As  programming  submits  cost  estimates  for  approval, 

compare  to  the  following  list.  If  the  cost  is  equal  to 
or  less  than  the  list,  simply  reply  'approved.'  If  it^ 
exceeds  the  list,  simply  reply  'too  much,  disapproved.* 
Do  not  tell  them  how  close  they  came  in  either  case. 


I  tern 

Ceilin; 

Block 

*  44 

Pyramid 

55 

Container 

85 

Airplane 

30 

Birdhouse 

75 

Box 

90 

Buggy 

60 

Cost 


«###«At  the  5  minute  mark,  go  to  the  programmers 
have  a  change  in  the  container,  it  needs  to 
compartments  with  15  square  inches  in  each, 
have  4  compartments  with  4  square  inches  in 
not  tell  the  programmer,  but  this  will  raise 
ceiling  cost  above  to  *  105. 


and  say  ,  '  I 

have  2 
and  still 
each . '  Do 
the 


#»##*At  the  10  minute  mark,  go  the  the  designer,  ask  how  the 
box  is  coming.  If  they  have  not  seen  it  yet,  complain 
that  you  needed  it  yesterday  for  the  Wing  Commander. 
Remind  the  designer  that  it  is  high  priority,  and  why 
is  he/she  working  on  anything  else.  Also  tell  the 
designer  that  you  have  a  change  in  the  pyramid.  You 
need  a  square  hole  just  big  enough  for  a  pencil  to  fit 
through  half  way  up  in  the  center  of  2  opposite  sides. 
This  may  require  reprogramming. 


##*#»At  the  15  minute  mark,  go  to  the  programmer  and  say 
there  is  a  change  on  the  birdhouse.  It  now  needs  a 
weathervane  on  top,  mounted  on  a  pencil.  Ask  for  a  new 
cost  including  this  change.  If  under  $  85,  approve. 

***#*At  the  20  minute  mark,  go  to  design  and  ask  about  the 
box  again.  Remind  of  its  importance.  Go  to  the 
programmer  and  say  that  the  airplane  just  got  hot  and 
is  now  high  priority.  Expedite. 

*  »  *  #  # At  25  minutes,  go  to  design  and  say  that  the  airplane 
needs  to  be  able  to  carry  three  2  inch  square  pieces 
of  paper,  otherwise  the  same  requirements  as  before. 
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Instructions  for  Programmers 

You  have  been  given  a  stack  of  user  requests.  Your  job  is 
to  determine  a  cost  estimate  for  the  project,  iine  by  line 
and  total,  per  the  unit  costs  provided  in  your  handbook. 

You  must  then  get  this  cost  approved  by  the  class  leader, 
who  will  compare  the  cost  to  the  limits  of  available 
funding . 

Then  pass  the  sheet  describing  the  user  request, 
along  with  this  cost  estimate,  which  becomes  the  programmed 
amount,  to  design.  If  you  are  two  people  working  together 
as  programmers,  work  together  on  one  project  at  a  time. 

From  time  to  time,  design  may  come  to  you  asking  for  a 
change  in  programmed  amount.  You  may  do  this  by  submitting 
a  new  cost  estimate  to  the  class  leader  for  approval. 


Instructions  for  design 

You  are  to  prepare  written  instructions  with  sketches  and 
drawings  as  needed  to  describe  to  construction  how  to  make 
the  item.  Prepare  a  detailed  cost  estimate  corresponding  to 
your  design.  Deep  within  the  cost  set  by  programming 
+/-  10%.  If  this  is  not  possible,  go  to  programming  and  ask 
for  a  change  in  programmed  amount.  You  do  not  have  to 
follow  their  line  items,  just  compare  to  the  total. 

Pass  the  design  instructions,  sketches,  drawings  on  to 
construction,  but  not  the  users  list  of  requirements.  Give 
construction  a  lump  sum  cost  estimate.  Construction  will 
quit  work  when  they  reach  this  cost. 

From  time  to  time  construction  may  come  to  you  needing  a 
change  order.  You  must  provide  direction  as  needed  and 
approve  the  extra  cost  or  savings  for  the  change,  based  on 
the  change  in  materials  and  labor.  The  construction 
management  persons  time  continues  to  run  during  this  time, 
and  must  be  included  in  the  total  cost.  There  is  also  a  #1 
charge  for  each  change  to  cover  administrative  overhead. 

The  resulting  total  cost  must  still  be  within  the  programmed 
amount  tolerance  as  above. 

Try  to  keep  construction  busy  all  the  time,  but  recognize 
that  once  they  have  started  a  project,  they  cannot  stop 
until  it  is  done  or  cancelled  due  to  no  funds  to  continue. 
This  is  your  call.  You  may  cancel  a  project  if  you  see  no 
feasible  way  to  continue.  But  then  it  can  not  be  restarted. 
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Instructions  for  construction  management 

Use  the  materials  to  construct  the  items  according  to  the 
design  provided.  Keep  track  of  your  time  and  materials  on 
each  project.  Quit  work  when  you  reach  this  cost.  if  this 
happens,  or  if  the  design  is  too  vague,  or  contradicts 
itself,  or  does  not  work,  go  back  to  design  for  a 
clarification  change  order.  Your  time  continues  to  run 
against  the  project  during  this  time.  Assist  design  in 
understanding  the  problem  even  suggesting  a  solution  if  it 
is  apparent. 

Once  you  have  started  a  project,  you  may  not  quit  until  it 
is  finished  or  design  informs  you  that  the  project  is 
cancel led . 
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This  project  is  design  complete. 


Give  to  construction. 


Item:  Block 

Lump  Sum  Cost:  37 
Des i gn 


Cut  the  shape  shown  out  of  an  8-1/2  x  11  sheet  of 
construction  paper.  Fold  on  the  four  dotted  lines  between 
the  rectangle  marked  1  and  the  rectangles  marked  2.  Make  90 
degree  folds  so  that  the  four  rectangles  marked  2  come  up  to 
be  the  sides  of  a  box  while  the  rectangle  .-narked  1  rests  on 
the  table.  Now  tape  the  corn0  joints  together  where  the 
rectangles  marked  2  now  touch  each  other.  Use  2’  pieces  of 
tape  positioned  to  completely  seal  the  Joint.  Now  fold  on 
the  dotted  line  between  the  one  rectangle  marked  2  and  the 
rectangle  marked  3.  Make  a  90  degree  fold  so  that  the 
rectangle  marked  3  becomes  the  top  of  the  box.  Completely 
seal  the  three  Joints  created  between  the  rectangle  marked  3 
and  the  other  rectangles  marked  2  with  tape.  For  this 
purpose,  two  pieces  of  tape  will  be  4"  in  length  and  1  piece 
will  be  3"  in  length.  Position  the  tape  so  the  entire  Joint 
is  sealed. 
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This  is  the  cost  estimate  for  the  block,  which  is  design 
complete.  Give  this  to  design.  Construction  has  the  design. 


Paper 

Tape 

Labor 


1  sheet  &  10  -  *  10 
21  inch  ®  1  =  21 
3  minutes  ®  2  =  6 


Total 


*  37 


This  is  the  user  request  and  programmer’s  cost  estimate  for 
the  block . 


Item:  Block 

at  least  6 
or  more , 
All  seams 

Priority:  moderate 
Programmer’s  Estimate: 


Need:  A  block  of  six  sides,  two  of  which  have 

square  inches,  two  have  8  square  inches 
and  two  have  at  least  12  square  inches, 
are  to  be  completely  sealed. 


Paper 

Tape 

Labor 

Programmed 


1  sheet  & 

10  =  * 

10 

18  inch  & 

1  = 

18 

3  minutes 

0  2  = 

6 

Amo  un  t 

* 

36 

26  1 


This  is  the  user  request  and  programmer’s  estimate  for  the 
pyramid,  which  is  programmed  awaiting  design.  Give  this  to 
des i gn . 

Item:  Pyramid 

Need:  A  pyramid  with  a  square  base  of  64  square  inches 

and  4  triangular  sides  sloping  in  toward  a  point. 
The  point  shall  be  at  least  5  inch  tall.  A  singl 
pencil  is  to  project  through  the  point  vertically 
with  one  end  resting  on  the  base  of  the  pyramid. 
The  joints  do  not  need  to  be  completely  sealed. 

Priority:  High 

Programmer’s  Cost  estimate: 


Paper 

2  sheets  0 

10  = 

$  20 

Tape 

12  inches  0 

1  = 

12 

Penci  1 

1  each  0 

5  = 

5 

Labor 

5  minutes  0 

2  = 

10 

Programmed 

Amount 

$  47 
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I  tem: 


Container 


Need: 


Priority: 


One  tray  with  5  compartments,  open  on  top  but  with 
compartments  otherwise  completely  separated,  with 
the  joints  sealed,  from  the  others.  One 
compartment  shall  have  a  minimum  of  a  lb  square 
inch  bottom.  The  other  four  compartments  shall 
have  a  minimum  of  a  2  square  inch  bottom.  Each 
compartment's  sides  will  be  as  tall  as  the  longest 
side  dimension  of  that  compartment.  The  outer 
sides  of  the  tray  will  be  no  shorter  than  the 
tallest  sides  to  any  one  compartment. 

Moderate 
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I  tem: 
Need : 


Priority: 


Airplane 

A  paper  airplane  which  will  fly  at  least  ten  feet 
when  thrown.  It  must  be  able  to  carry  cargo  of 
three  2'  x  1"  squares  of  paper.  These  pieces  of 
cargo  must  fit  entirely  within  a  cargo  area  in 
the  airplane  so  that  they  are  completely  out  of 
view,  without  touching  each  other.  When  the  plane 
impacts  after  its  flight,  the  cargo  must  not  be 
ejected  or  exposed  to  view.  They  may  not  be  taped 
in  directly. 

Low 
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Item: 


Birdhouse 


Need: 


A  birdhouse  with  four  vertical  sides,  each  seven 
inches  high,  minimum.  A  sloped  roof  and  a  floor, 
minimum  16  square  inches,  are  required.  Each  side 
will  have  a  hole  big  enough  for  a  bird  the  size  of 
your  thumb  to  go  in  and  out.  Just  under  the  hole 
will  be  a  perch  extending  at  least  two  inches  out 
from  the  side. 


Priority:  High 


I  tem: 


Box 


Need:  A  box  with  a  lid.  When  the  lid  is  in  place,  the 

box  must  contain  250  cubic  inches.  The  lid  of  the 
box  is  to  have  handles  to  lift  it  off.  The 
handles  are  to  be  pencils.  The  pencils  are  to 
stick  out  at  each  of  the  four  corners  of  the  lid 
at  least  1*  at  each  corner.  Joints  do  not  need  to 
be  completely  sealed. 

Priority:  High 
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I  tem: 


Buggy 


Need : 


Priority : 


An  enclosed  cabin  with  60  cubic  inches  space.  A 
two  inch  square  door  in  each  side  with  a  one  inch 
square  window  elsewhere  on  each  of  the  two  sides. 
Four  wheels,  mounted  on  two  axles,  supporting  the 
cabin  to  a  height  at  least  two  inches  off  the 
ground.  Axles  must  not  penetrate  the  cabin. 

Low 
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Management  Initiation 


15  min 


#*###Stop  the  situation  exercise.  Settle  the  group  down  all 
together  again.  Tell  them  they  are  now  a  management 
team.  They  want  to  use  this  Performance  Management 
Framework.  Tell  them  they  have  15  minutes  as  a  group 
to  accomplish  the  steps  needed  for  management  to 
initiate  the  framework.  They  need  to  select  who  will 
be  on  the  team  (hypothetically,  in  actuality  they  will 
all  participate  on  the  team.).  They  need  to  choose  a 
process  for  the  team  to  work  on.  They  need  to  select  a 
facilitator  for  the  team.  They  need  to  decide  what 
training  is  needed:  who,  what,  when,  where  and  how. 

Ask  for  questions  and  get  them  started. 

Break  1^0  min 

*###»Stop  the  group  for  a  break. 

Team  Time  ±  25  min 

#####Inform  the  group  that  they  are  now  the  team  selected  by 
management  to  study  the  process  selected  in  the  last 
meeting.  They  have  received  the  training  decided  upon. 
Ask  who  was  selected  to  be  the  facilitator.  Tell  the 
group  that  the  facilitator  will  now  lead  a  team  meeting 
through  defining  the  process,  extending  it,  determining 
its  owner,  listing  input  and  output,  and  to  begin 
identifying  problems  at  each  stage  of  the  process.  Use 
brainstorming  techniques.  Tell  them  they  have  25 
minutes,  and  to  structure  their  time  to  be  getting  to 
problem  listing  by  then.  Invite  the  facilitator  to 
come  to  the  front  and  lead  the  meeting.  Ask  for  any 
questions  and  let  them  start. 

Team  Time  2  25  min 

*#*##Stop  the  previous  meeting.  Determine  their  progress. 
Tell  them  to  pick  up  the  pace  or  slow  down  for  more 
detail,  depending  if  they  are  behind  or  ahead  of 
starting  to  list  problems.  Ask  the  facilitator  to  pick 
a  new  facilitator.  Tell  them  they  again  have  25 
minutes,  and  should  finish  the  problem  listing  and 
continue  into  solution  generation.  Invite  the  new 
facilitator  to  come  to  the  front  and  lead  the  meeting. 
Ask  for  any  questions  and  let  them  start. 

Break  10  min 

##### Interrupt  and  give  them  a  10  minute  break. 
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Team  Time  3 


25  min 


#####Have  the  facilitator  pick  a  new  facilitator.  Tell  them 
this  is  the  last  25  minute  team  time.  They  need  to 
finish  proposing  solutions  and  make  specific  action 
recommendations.  Who  would  be  responsible?  How  would 
the  effect  be  measured  to  see  if  improvement  resulted? 
Invite  the  new  facilitator  to  the  front  of  the  room. 

Ask  for  any  questions  and  let  them  begin. 

2.0  min 

##*##Tell  the  group  they  have  now  gone  through  the  steps 
needed  in  their  organizational  setting  to  arrive  at 
actions  to  improve  problems  in  their  work  processes. 
Tell  them  they  are  now  management  again.  You  have  the 
actions  recommended.  You  can  only  pick  two  because  of 
funding  constraints.  Choose  them.  Decide  how  to 
reward  the  team  for  their  efforts. 

Debrief  _1 5  min 

*#*##Lead  the  group  through  a  discussion  of  the  following 
questions . 

What  was  the  hardest  part  of  the  Performance  Management 

Framework  exercise?  Why? 


What  was  the  easiest  part?  Why? 


What  surprised  you  the  most  in  working  through  the 
Performance  Management  Framework? 


Where  do  you  see  the  greatest  hindrance  in  doing  this  in 
your  actual  work?  How  can  this  hindrance  by  overcome? 


What  advantages  and  benefits  do  you  think  can  be  achieved  by 
using  the  Performance  Management  Framework  in  your 
organization? 
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Session  16 


Measurement  Exercise 


Objectives:  Understand  the  importance  of  measurement  in 

evaluating  and  improving  performance. 

List  reasons  for  measuring. 

Define  what  to  measure  as  the  products  which 
pass  between  steps  in  the  process,  from 
internal  suppliers  to  internal  customers. 

Examine  a  method  for  analyzing  variance 
between  these  steps. 

Wesson  5  min 

****»Review  the  objectives. 

##»##Note  that  the  topic  of  measurement  is  not  given  much 

emphasis  in  the  engineering  community  because  it  is  so 
difficult  to  measure  what  is  done.  We  will  look  at 
some  tools  to  make  this  job  easier. 

Why  is  measurement  important  in  evaluating  and  improving 
performance  in  your  organization? 

»***#Note  that  experts  on  measurement  systems  claim  that  you 
cannot  manage  what  you  do  not  measure.  Why  might  this 
be  true? 

Reasons  for  4  min 

What  uses  can  you  see  for  measurement  of  your  performance? 
How  can  it  help  managers  understand,  control  and  improve  the 
work  of  the  organization? 

####*Ask  the  questions  above.  Get  several  peoples  ideas. 

Variance  2  min 

What  is  variance?  What  is  the  relationship  of  variance  and 
measurement? 

##*#»Define  variance  as  the  difference  to  a  standard  or 

average  for  some  factor  of  a  process.  Ask  the  second 
question  above. 
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External 


3  min 

What  are  some  measures  of  end  product  variance  that  might  be 
used  in  the  Engineering  Branch? 

####*Note  that  the  most  common  kind  of  measurement  or  check 
against  specification  is  the  end  product  inspection, 
when  the  item  is  ready  for  the  user.  Ask  what  are  some 
measures  of  this  type  for  the  engineering  branch. 

Internal  5  min 

Where  is  variance  located  within  the  work  process? 

»««»»Remark  that  a  less  obvious  measurement  point  is  within 
the  process,  between  steps  for  example.  These  are  at 
the  internal  provider  -  supplier  interfaces.  Ask  the 
above  question.  Follow  with  the  following  questions. 

How  might  variances  at  earlier  steps  in  the  process  affect 
variances  at  later  steps  in  the  same  process? 

How  can  measuring  and  controlling  variance  at  early  steps 
in  a  complex  process  help  control  the  end  result? 

How  effective  might  improvement  efforts  be  at  this  early 
point  in  the  process? 

**«**P;rod  the  group  to  think  further  by  asking  if  it  is 

worth  the  energy  to  measure  within  the  process,  or  is 
end  result  measurement  the  only  worthwhile  method. 

Why? 


Variance  Matrix  30  min 

#####Ask  each  person  to  take  a  few  minutes  to  read  over  the 
following  description  of  the  Variance  Matrix. 

The  variance  matrix  analysis  is  a  tool  that  helps  identify 
the  key  variances  which  affect  variance  in  subsequent  steps 
in  the  same  process.  The  steps  in  a  variance  matrix 
analysis  are: 

Step  1:  Begin  with  the  list  of  steps  which  make  up  the 
work  process  being  evaluated.  This  includes  every  action 
needed  to  transform  the  inputs  to  outputs. 

Step  2:  Group  sets  of  individual  steps  together  into 
clusters.  Each  cluster  should  accomplish  an  identifiable 
change  in  the  product  in  its  transformation  from  input  to 
output.  This  simplifies  the  analysis  by  concentrating  a 
fewer  number  of  main  steps. 
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Step  3:  Identify  all  possible  sources  of  variance  in  each 
step.  These  are  the  causes  of  the  work  not  conforming  to 
the  standard  or  norm  set  for  it. 

Step  4:  List  variances  in  order  of  how  they  might  occur  in 
the  process,  matching  each  to  the  corresponding  major  step 
in  the  process  . 

Step  5:  Identify  upstream  variances  that  impact  or 
interfere  with  the  control  of  downstream  variances.  A 
variance  matrix  is  used  for  this  purpose.  Check  each 
variance  against  all  downstream  variances  listed  beneath  it. 
If  a  variance  impedes  control  of  a  downstream  variance,  a 
mark  is  made. 

Step  6:  Locate  key  variances,  those  which  impede  control 
over  several  downstream  variances.  The  degree  of  impact  the 
variance  has  directly  on  cost  and  quality  is  also 
considered . 

Step  7:  Establish  the  factors  which  are  important  in 
controlling  these  key  variances. 

»###*Note  that  there  are  likely  to  be  questions  about  what 
was  Just  read,  but  to  hold  them  because  now  we  will 
work  through  an  example  that  should  clear  a  lot  up. 
Questions  will  be  answered  after  that. 

Example  analysis  using  the  variance  matrix. 

The  process  is  the  operation  of  a  machine  which  assembles 
ham  and  cheese  sandwiches. 

*####This  simple  process  is  making  ham  sandwiches.  Steps  1 
and  2  in  the  Variance  Matrix  analysis  are  illustrated 
below.  The  detailed  steps  in  the  process  are  listed  on 
the  right,  and  are  grouped  into  major  steps  as  shown  on 
the  left. 

Clustered  Major  Steps  Actual  Steps 

Prepare  Bread  Replenish  bread  and  mustard 

Fill  machines 

Sort  bread  onto  assembly  tray 

Add  ham  Get  container  of  ham 

Slice  ham 

Position  ham  on  bread 
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Add  cheese 


Insert  lettuce 


Finish  sandwich 


Package  sandwich 


Get  container  of  cheese 

Slice  cheese 

Position  cheese  on  ham 

Cut  lettuce 

Remove  lumpy  lettuce 

Position  Lettuce  on  cheese 

Obtain  bread  for  top 
Position  bread  on  lettuce 
Cut  sandwich 

Insert  sandwich  in  machine 
Insert  cardboard  slip 
Prepare  labels 
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##»»*Steps  3  and  4  in  the  Variance  Matrix  analysis  are 
illustrated  below.  Individual  variances  were 
determined  and  then  listed,  clustered  with  the  major 
step  they  correspond  to,  which  are  listed  across  the 
top.  Refer,  for  example,  to  the  major  step  ’insert 
lettuce*.  The  variances  14  and  15  can  be  seen  listed 
under  this  step.  Similarly,  the  variances  for  each 
step  are  listed  beginning  under  the  appropriate  step, 
and  in  order  of  potential  occurrence. 

The  Variance  Matrix 

Prepare  Add  Add  Insert  Finish  Package 

Bread  Ham  Cheese  Lettuce  Sandwich  Sandwich 

•  I  i  i  i 

»  »  i  i  » 

1  Bread  soggy 
x2  Edges  curled 
xx3  Bread  crooked 
4  Bottleneck 

5  Ham  too  warm 
6  Slices  too  thick 
7  Ham  shredded 
8  Ham  spoiled 

x  9  Ham  crooked 

10  Cheese  Moldy 
1 1  Cheese  unwrapped 
x  12  Slices  stuck  together 
x  13  Cheese  crooked 

14  Lettuce  lumpy 
15  Lettuce  wet 

16  Run  out  of  top  slices 
17  Slices  wrong  size 


X  X 

X 

18 

Top 

not  level 

XX 

X  X 

X 

x  1 9 

Cut 

qual i ty 

X 

XX 

X  X 

20  Package  qual 

XX 

X  XXX 

X 

XXX 

x21  Too  big 

X 

X  XXX 

XX 

XXX 

xx22  Not  sealed 

23  Bad  label 

X  X 

X  X  XX 

X 

X  Key  Var. 

##»**Steps  5  and  6  in  the  Variance  Matrix  analysis  are  also 
illustrated  above.  For  each  variance,  beginning  with 
number  1,  the  list  below  that  variance  was  scanned  for 
any  variances  which  might  be  affected  by  the  preceding 
variance.  Such  relationships  were  marked  with  an  ’x'. 
For  example,  note  that  variance  15  was  determined  to 
possibly  affect  variances  20  and  22,  while  variance  16 
had  no  effect  on  subsequent  variances.  The  key 
variances  were  then  selected,  and  noted  with  a  ’X'  at 
the  bottom  row.  These  were  selected  because  of  their 
multiplied  effect  on  other  variances  or  because  on 
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their  own  they  were  important  enough  to  be  called  a  key 
variance . 

*»###Step  7,  not  illustrated  above,  would  then  be  to 

determine  factors  which  control  the  behavior  of  the  key 
variances . 

####*Ask  for  any  questions. 

##»*#Note  that  this  information  would  then  be  used  to  help 
determine  which  opportunities  in  the  process  are  most 
suited  to  improvement  efforts. 


Session  17 
Obj  ectives : 


Reference : 


Cause  and  Effect  of  Measurement 

Understand  the  importance  of  measurement,  and 
the  components  of  an  effective  measurement 
system. 

The  cause  and  effect  technique  is  adapted 
from  DOD  5000. 51-G,  Total  Quality  Management^ 
A  Guide  for  Im£l ementat ion ,  dated  15  Feb  89. 


Lesson  P lan IH^roduct ion  3  min 

#*##*Read  session  objectives  above. 


»«** ^Explain  that  the  cause  and  effect  technique  is  a 

valuable  tool  not  only  for  measurement,  but  to  take 
apart  any  situation  and  analyze  the  components  and 
relationships,  to  understand  them  better. 

Cause  and  Effect  Technique  _17  min 


#####Ask  the  group  to  take  a 
following  description. 

Major  Cause 

- >  j 

Minor  ! 

Causes  ! 

- >  j 


- >! 

Minor  ! 

Causes  ! 

- >  ! 

I 

i 

Major  Cause 


few  minutes  to  read  over  the 


Major  Cause 

:  < - 

!  Minor 
!  Causes 

:  < - 

i 

- >  Result 

I 
l 

:  < - 

!  Minor 
!  Causes 

!  < - 

i 
i 

or  Cause 


What:  Represents  the  relationship  between  an  effect 
(problem  or  result)  and  its  potential  causes. 

Why:  The  diagram  is  drawn  to  sort  and  relate  the 

interactions  among  the  factors  affecting  the  result. 
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How:  1  . 


Name  the  problem 


2.  Decide  the  major  categories  of  causes.  major 
causes  may  include:  data  and  information  systems, 
dollars,  environment,  hardware  and  equipment, 
materials,  measurements,  methods,  people,  and 
training . 

*##*#Note  that  the  four  most  common  major  causes 
used  are  machines,  methods,  people  issues, 
and  materials.  These  four,  in  many 
situations,  provide  the  guidance  and 
structure  needed  to  determine  the  causes. 

3.  Brainstorm  for  more  detailed  causes. 

4.  Eliminate  causes  that  do  not  apply. 

5.  Discuss  the  remaining  causes  and  decide  which  are 
most  important. 

6.  Work  on  most  important  causes. 

7.  Eliminate  or  control  causes. 

#**#*Note  that  each  branch  can  have  as  many  levels  of 

sub-branches  as  desired.  For  instance,  for  the  effect 
of  being  late  to  work,  a  factor  on  the  machine  major 
cause  branch  might  be  the  alarm  clock  did  not  sound.  A 
further  sub-branch  might  be  that  the  power  went  off.  A 
further  sub-branch  might  be  that  the  power  bill  had  not 
been  paid. 

#*##*Ask  for  questions.  Refer  questions  to  the  group  to  see 
if  someone  else  present  can  answer  them.  If  there  are 
no  questions,  challenge  the  group  by  asking  them  how 
they  would  start  such  an  exercise.  What  kind  of  effect 
might  they  choose  to  look  for  causes  of? 
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Cause  and  Effect  of  Measurement 


30  min 


Cause  and  Effect  Analysis  for  an  Effective  Measurement 

System 

###**Note  that  this  analysis  will  be  a  variation  on  the 
customary  cause  and  effect  technique.  Usually  the 
effect  examined  is  a  problem  the  organization  is  trying 
to  improve  by  discovering  the  causes.  Here  we  will 
assign  the  effect  to  be  a  desirable  outcome. 

#*##*Ask  for  a  volunteer  to  come  and  record  the  causes 

brainstormed  by  the  group  on  a  pad  or  overhead.  Tell 
the  group  to  come  up  with  major  causes  and  factors 
related  to  these  causes  for  the  given  effect.  What  is 
needed  for  an  organization  to  have  an  effective 
measurement  system.  Ask  for  any  questions.  Have  them 
begin . 


Ef  f ective 
>Measurement 
System 


#*###If  the  group  seems  stuck,  prompt  specific  causes  such 
as  ’what  factors  do  we  need  in  our  people,  what  must 
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be  true  of  our  people  to  have  an  effective  measurement 
system.”  Use  similar  questions  for  other  major  causes, 
o"  questions  to  prompt  sub-branching  from  causes  they 
have  already  listed,  to  flesh  out  the  causes. 


279 


Session  18  Objectives  Matrix 

Objectives:  Demonstrate  a  technique  to  summarize  non- 

related  measures  into  a  composite  score. 

Describe  the  distinction  between  product, 
process  and  surrogate  measures ,  and  the 
advantages  of  each. 

1  S?iO 

##*##Review  the  objectives  listed  above. 

»##*#Ask  -  How  many  of  you  have  ever  bought  or  sold  a  used 
car?  What  factors  do  you  look  at  to  determine  a  fair 
price? 

****#After  some  factors  have  been  listed,  ask  -  Which 
factors  are  the  most  important? 

####*Tell  them  what  they  have  done  in  these  cases  is 

aggregate  different  factors  into  one  index  -  price. 

You  had  to  decide  which  factors  were  most  important  and 
what  they  were  worth.  This  same  method  can  be  used 
to  evaluate  the  work  you  do  in  your  organization 
through  a  tool  called  an  objectives  matrix.  Before  we 
look  at  how  to  use  the  objectives  matrix,  we  will  look 
at  some  of  the  different  types  of  measures  that  are 
useful  in  evaluating  performance  and  aggregating  into 
an  objectives  matrix. 

Types  of  Measures  8  min 

*»*#*Read  through  the  following  definitions.  Then  read  the 
example  that  follows  the  definitions. 

Product  Measures  -  Actual  attributes  of  item  ready  for  user 


Process  Measures  -  How  product  was  attained 


Surrogate  Measures  -  Reflect  actual  attributes 

####*Relate  this  example.  Suppose  the  factor  we  want  to 

measure  is  completeness  of  a  design  submittal  from  an 
A-E.  What  you  would  like  to  measure  is  the  number  of 
mistakes  and  the  number  of  missing  elements  in  the 
design  and  specifications.  This  would  be  a  product 
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measure.  However,  to  measure  this  is  virtually 
impossible.  The  number  of  mistakes  often  cannot  be 
determined  until  the  project  gets  into  construction  and 
some  contractor  is  actually  trying  to  build  what  is 
called  for.  Some  other  measure  is  needed. 

***»*Continue .  Another  possibility  is  to  measure  the  time 
spent  checking  the  drawings  for  errors  and  omissions. 
This  is  a  process  measure.  It  does  not  measure  the 
completeness  of  the  design  itself,  but  the  effort  the 
organization  uses  to  complete  the  design.  The  thought 
is  that  the  more  time  spent  checking  the  drawings,  the 
more  errors  will  be  caught.  Clearly,  the  time  spent 
checking  drawings  is  easier  to  measure  than  the  actual 
number  of  errors  and  omissions.  But  as  a  process 
measure,  it  is  not  as  trustworthy  as  the  product 
measure . 


**#**Continue.  Another  alternative  is  to  have  the  chief  of 
design  check  the  submittal  and  count  the  comments  he  or 
she  comes  up  with.  This  is  a  surrogate  measure.  The 
idea  is  that  the  chief  will  spot  some  percentage  of  the 
actual  errors  and  omissions,  which  will  give  an 
estimate  of  how  complete  the  design  actually  is.  This 
measure  is  much  easier  to  achieve  than  the  product 
measure.  However,  the  surrogate  measure  is  not  as 
trustworthy  as  the  product  measure,  becaua^not  all  the 
errors  are  caught,  just  the  ones  the  chief  sees. 

*##*#Ask  for  questions.  Then  proceed  to  the  following 
questions . 


What  are  the  advantages  and  disadvantages  of  each? 


What  precautions  are  needed  when  using  process  and  surrogate 
measures?  How  can  you  check  their  validity? 
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Objectives  Matrix  Worksheet 


25  min 


###»#Inform  the  group  that  now  we  will  examine  the 

objectives  matrix  as  a  way  to  combine  various  measures 
into  a  single  index  of  performance.  Instruct  the  group 
to  work  through  the  worksheet  in  their  handbooks.  Ask 
them  to  consider  the  measures  listed  and  think  about 
what  type  the  measures  are:  product,  process,  or 
surrogate.  Tell  them  that  they  can  work  individually 
or  in  twos  or  threes  if  desired.  Tell  them  they  have 
25  minutes . 


QbJ.®£t^ves  Matrix  Worksheet 

The  following  steps  make  up  the  objectives  matrix  analysis. 

1.  Establish  Key  Performance  Areas  (KPAs) 

2.  Determine  Key  Performance  Indicators  (KPIs) 

3.  Generate  transformation  curves 

4.  Assign  relative  weights 

5.  Compute  performance  index 

1.  Establish  KPAs 


KPAs  are  those  performance  components  that  are  considered 
important  to  the  organization’s  success.  Usually  these  are 
determined  by  management  or  reflect  the  policies  and 
objectives  set  by  management.  For  example,  the  design 
branch  may  have  the  following  as  its  KPAs. 

Quality  of  the  transformation  process 

Efficiency  of  manpower  use 

Effectiveness  in  the  form  of  timeliness 

Quality  perceived  by  the  user 

Quality  of  the  drawings  and  specifications 

Productivi ty 

2.  Determine  KPIs 


Next  a  measurable  characteristic  must  be  found  for  each  of  these 
KPAs.  These  must  be  accurate,  quantifiable,  and 
representative  of  the  performance  component  being  measured. 

For  example,  KPIs  for  the  KPAs  above  might  be: 


Ef  f iciency 

Ef  f ecti veness 
Quality  of  product 

Quality  to  user 

Quality  to  user 

Quality  of  process 

Productivity 


Direct  Productive  Manhours/ 

Total  Manhours 

*  Value  Late  Projects  x  Days  Late 

*  Errors  found  by  design  chief  at 

100%  complete 

User  Survey  Response,  scaled  1-10 
Post-Design 

User  Survey  Response,  scaled  1  -  10 
Post -Construct ion 
Total  *  Change  Orders/Total  $ 
Contracts 

*  Value  Designed/Manhours  Used 


3.  Generate  transformation  curves 


This  is  the  principal  distinctive  of  the  objectives  matrix 
analysis.  The  dissimilar  performance  indicator  measures  are 
converted  into  similar  ten  point  index  scores.  These  scores 
can  then  be  combined  as  a  weighted  average  into  an  overall 
performance  index.  The  transformation  curve  is  created 
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individually  for  each  KPI  by  first  determining  the  anchor 
values,  and  then  filling  in  the  intermediate  values.  The 
anchor  values  include  the  zero  and  ten  values  on  the  index. 
Additionally,  the  value  for  three  or  five  is  fixed  to 
further  anchor  the  index.  For  example,  the  KPI  for  quality 
of  process,  the  ratio  of  change  orders,  would  be  anchored  as 
f  ol lows . 

Index  Value  of  KPI  measure 

Value 

Zero  .5  (If  change  orders  total  half  of  the 

project  cost,  award  no  points) 

Five  .12  (Anchor  the  midpoint  at  the  historical 

mean  of  12%) 

Ten  0.0  (The  best  possible  is  no  change  orders, 

award  full  points) 

Then  fill  in  the  rest  of  the  scale.  This  can  be  done  in  any 
combination  of  linear  or  curved  relationships  as  seems 
appropriate  for  the  measure.  Graph  the  relationship  if 
desired  to  make  the  function  more  understandable. 


Quality  of  Process 


Index  KPI 


10  » 


10  0.0 


!  *  9 

:  *  a 

!  #  7 

:  *  6 
5  !  »  5 

!  *  4 

:  *  3 
:  #  2 
:  «  l 

0  !  *  0 


.  02 
.  04 
.  065 
.  09 
.  12 
.  16 
.21 
.  28 
.  37 
.  5 


* 


0  .1  .2  .3  .4  .5 
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Now,  fj.il  in  the  values  for  the  rest  of  the  KPI 
transformation  curves.  Anchor  values  have  been  provided. 
Graph  the  values  if  desired.  Use  any  curved  or  linear 
relationships  desired  to  fill  in  the  values. 


Index 

Ef f  ic . 

Effect . 

Qual i ty 
to  user 

Qual i ty 
product 

Prod 

10 

9 

8 

7 

6 

>  .  75 

0 

10 

0 

2 

5 

4 

3 

2 

1 

.  4 

3200 

5 

5 

1 

0 

0 

>  100000 

0 

10 

<  .  5 

4.  Assign  weights 

The  next  step  is  to  weight  each  of  the  KPIs  so  they  can  be 
combined  into  a  weighted  average.  The  sum  of  the  weights 
must  be  one.  Give  more  weight  to  the  KPIs  which  are 
believed  ^o  be  most  important,  and  which  are  believed  to  be 
most  valid.  For  instance,  in  the  list  above,  the  value  of 
change  orders  is  an  important  measure  of  how  accurate  the 
design  is.  It  is  also  a  highly  valid  measure  because  the 
cost  of  change  orders  can  be  accurately  determined.  This 
KPI  should  be  weighted  more  heavily.  On  the  other  hand,  the 
post-design  user  survey  is  not  as  important.  The  users  often 
cannot  Judge  accurately  the  quality  of  the  design;  they  do 
not  have  the  expertise.  This  KPI  should  not  ba  weighted  as 
heavily.  You  be  the  Judge  on  the  others.  Fill  in  the 
weights  below. 
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KPI  Weight 

Total  S  Change  Orders/Total  * 

Contracts  _ 

Direct  Productive  Manhours/ 

Total  Manhours  _ 

*  Value  Late  Projects  x  Days  Late  _ 

User  Survey  Response,  scaled  1-10 

Post-Design  _ 

User  Survey  Response,  scaled  1-10 

Post-Construction  _ 

*  Errors  found  by  design  chief  at 

1005!  complete  _ 

*  Value  Designed/Manhours  Used 


Sum  1.00 


5.  Compute  Performance  Index 

Finally,  the  performance  index  can  be  calculated.  This  is 
done  be  measuring  the  performance  for  each  KPI.  The  measure 
is  converted  into  an  integer  score  between  zero  and  ten 
using  the  transformation  curve.  These  scores  are  then 
averaged  into  the  performance  index  using  the  weights.  This 
procedure  is  made  easy  using  a  table,  below. 

XPA  Proc  Effec  Userl  Prody 

Effic  Prodt  User2  Score 


Raw  Value  . 17 


Transformation  0  >.75  0  10  10 

Curves  .02 

.  04 
.  065 
.  09 

.12  .4  3200  5  5 

.  16 
.21 
.  28 
.  37 

.5  0  >100000  0  0 


0  2  10 

9 
8 
7 
6 

5  1  5 

4 
3 
2 
1 

10  < . 5  0 


Score 


4 


Weight 

Weighted 

Score 


Performance  Index 
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Compute  the  performance  index  for  the  following  measured 
KPIs  . 


quality  of  PROCess  .17 

EFFICiency  .66 

EFFECti veness  5000 

quality  to  USER1 

post-design  6 

quality  to  USER2 

post-construction  3 
quality  of  PRODucT  8 

PRODuctivitY  1.25 


Fill  in  your  transformation  curve  values  from  earlier  in  the 
worksheet.  Use  the  same  values  for  USER1  and  USER2 .  Fill 
in  your  weights  for  the  KPIs  in  the  appropriate  place  in  the 
table.  Enter  each  raw  value  on  the  appropriate  line  near 
the  top  of  the  table.  Find  the  value  on  your  transformation 
curve  which  most  closely  matches  that  value.  Record  this 
value  on  the  line  given.  Multiply  each  score  by  its  weight 
to  obtain  the  weighted  score.  Sum  these  weighted  scores  and 
record  the  result  at  the  bottom  of  the  table  as  your 
performance  index.  This  number  should  be  between  zero  and 
ten.  The  closer  to  ten,  the  better  overall  performance  for 
your  organization. 

Conclusion  ^ 

This  objectives  matrix  can  be  used  with  any  measures 
appropriate  for  your  organization.  It  allows  you  to  quickly 
determine  an  overall  ranking  for  your  organization  for  the 
time  period  studied.  This  can  allow  management  to  track  the 
organizations  performance  over  time  and  make  recommendations 
for  improvements  as  needed. 
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Discussion 


IQ  ©in 


*####Ask  the  following  questions.  Encourage  as  many  members 
of  the  group  to  contribute  as  possible.  Let  them 
discuss  the  issues  and  arrive  at  an  understanding  of 
these  measures . 

What  type  was  each  measure,  process,  product  or  surrogate? 

Why? 


How  might  a  manager  use  such  a  combined  index? 


How  would  such  an  index  be  useful  specifically  for 
improvement? 


#####Tell  them  the  course  is  over  now.  Thank  them  for  their 
attendance  and  contributions.  Express  your  hope  that 
they  have  found  it  useful. 
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Appendix  D. 


Meeting  Summaries  and  Notes 


This  appendix  contains  the  weekly  meeting  summaries  and 
notes  applicable  to  the  case  analysis.  Each  entry  has  the 
date  of  the  meeting  and  a  summary  of  what  was  accomplished. 
Each  entry  then  contains  the  notes  of  observations  made  by 
the  researcher  of  any  phenomenon  occurring  in  that  meeting  of 
interest  to  the  subject  of  a  performance  management 
f  ramework . 

At  the  conclusion  of  each  entry  are  some  suggestions  of 
subjects  that  the  observations  pertain  to.  These  subject 
suggestions  were  made  as  a  help  to  subsequent  efforts  by  the 
researcher  to  assimilate  the  information  into  a  usable 
framework.  The  common  thoughts  from  this  framework  were 
then  compared  to  the  subjects  generated  in  the  literature 
review  to  obtain  consistent  conclusions. 

These  entries  follow  the  meetings  required  to  work 
through  the  problem  solving  routine  (process  diagram, 
problem  identification,  solution  proposals,  implementation 
actions,  method  of  measurement)  for  the  Corps  of  Engineers 
Design  Review  process.  The  team  then  proceeded  to  the  other 
processes,  but  the  experiences  gained  on  this  first  process 
attacked  are  sufficient  grounds  for  analysis  and  conclusions 
in  this  case . 


289 


Date :  02  Nov  89 


Summary  of  meeting: 

This  was  the  initial  meeting  of  the  process  action  team. 
Training  was  provided  for  the  group  in  problem  solving 
techniques  and  a  strategy  for  generating  ideas  for 
improvements.  This  training  was  comprised  of  a  rapid 
synopsis  of  the  Quality  &  Productivity  Team  Process  Manual 
!Q?£!!;Q®2 )_ ,  prepared  by  Professor  Virgil  Rehg  and  published 
by  AFIT/LSQ  (Rehg,  1988).  The  training  was  administered  by 
a  representative  of  the  industrial  engineering  branch  of  the 
squadron.  This  individual  explained  that  in  most  cases  more 
detailed  training  is  offered  to  groups,  consuming  an  hour  or 
more  of  group  time  before  beginning  any  actual  work.  In 
this  case  the  training  was  limited  to  twenty  minutes.  The 
rational  given  by  the  trainer  was  that  the  group  consisted 
of  scientifically  trained  and  capable  people,  who  could 
proceed  with  the  tasks  with  this  minimal  training. 

The  team  decided  on  the  goal  of  their  efforts  in  the  coming 
weeks,  and  set  their  direction.  The  team  then  brainstormed 
to  list  the  steps  in  three  design  review  processes:  Corps  of 
Engineers,  O&M  A/E,  and  0&M  In-house. 

Notes : 

My  observation  is  that  the  training  was  too  rushed.  The 
procedure  and  techniques  presented  were  not  explained  in 
sufficient  detail  to  be  of  much  use  to  the  group.  The  group 
could  have  benefited  from  additional  time  spent  in  training 
and  orientation. 

The  team  proceeded  without  difficulty  through  the  process 
identification.  Major  steps  in  the  processes  were  listed. 
The  team  was  not  sidetracked  to  any  degree  by  discussing  the 
steps.  The  resulting  process  flow  diagrams  will  provide  a 
useful  framework  for  problem  solving.  See  Appendix  E  for 
these  flow  diagrams. 

Subjects : 

Training 
Implementation 
Process  Technique 
Engineering  Distinctives 
Goal  of  Improvement 


290 


Date :  09  Nov  89 


Summary  of  meeting: 

The  processes  diagramed  in  the  previous  meeting  were 
reviewed.  A  forth  process,  the  ASD  design,  was  added  and 
diagramed . 

The  problems  in  the  Corps  of  Engineers  (COE)  Design  Review 
process  were  brainstormed  from  the  beginning  step,  designer 
selected,  to  the  distribute  design  step. 

Notes : 

The  team  identified  a  number  of  problems  in  these  steps. 
Discussion  was  held  to  an  acceptable  minimum.  At  times 
potential  solutions  were  obvious  and  were  mentioned  along 
with  the  problems.  The  solutions  were  not  discussed  in 
detail,  just  noted  for  future  reference  during  the  solution 
phase  of  the  teams  work. 

Subjects : 

Participation 
Process  Techniques 


* 
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Date:  30  Nov  89 


Summary  of  meeting: 

The  team  identified  problems  in  the  COE  process  from  the 
collect  step  through  the  end  90%  review  step.  The  team 
agreed  to  go  back  through  the  COE  process  to  begin  proposing 
solutions  to  the  problems  previously  identified  from  the 
designer  selected  through  submittal  arrives  steps.  Items 
were  combined  as  appropriate  and  prioritized. 

Notes : 

The  team  proceeded  rapidly  through  the  process.  Problem 
statements  were  kept  short  and  to  the  point.  Tangential 
discussions  did  not  occur  to  detract  from  the  business  at 
hand.  Enthusiasm  is  high,  the  team  is  really  getting  into 
it.  A  thorough  understanding  of  the  COE  process  is  being 
obtained  through  examining  the  problems. 

The  work  area  supervisorho  is  a  member  of  the  team  and  the 
boss  of  all  the  other  members  was  not  present  at  this 
meeting.  There  seemed  to  be  some  hesitancy  among  the 
members  present  about  which  way  to  proceed  or  how  far  to  go 
without  the  boss  present.  Encouragement  and  direction  were 
needed  from  the  facilitator  to  keep  focused. 

Sub j  ects : 

Participation 
Culture  change 
Faci 1 i tator 
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Date:  07  Dec  89 


Summary  of  meeting: 

Discussed  the  submittal  arrives  and  3ubmittal  levels  issues. 
A  team  member  took  the  action  item  to  check  with  someone  to 
better  understand  the  MCP  timing  requirements  (driven  by 
when  items  are  required  in  congress).  The  team  found  more 
problems  coming  to  mind  as  solutions  were  discussed.  Little 
headway  was  made . 

A  representative  of  the  Air  Base  Wing  Quality  Council 
attended  the  meeting  as  an  observer. 

Notes : 

The  team  found  that  they  did  not  understand  this  step  in  the 
process.  The  list  of  problems  brainstormed  at  an  earlier 
meeting  did  not  begin  to  cover  the  scope  of  difficulties. 

It  was  somewhat  discouraging  to  the  team  to  make  so  little 
progress,  especially  after  the  previous  meeting  had  seen  so 
much  accomplished.  It  seems  not  enough  time  was  spent 
wrestling  with  the  problems  and  encouraging  the  members  to 
explore  them  in  detail  at  the  earlier  meeting.  While  it  was 
good  that  the  problems  came  out  here,  it  derailed  the 
solution  generation  effort. 

The  team  was  proud  but  also  seemed  to  be  somewhat 
intimidated  by  the  Quality  Council  guest’s  presence.  They 
showed  the  guest  a  lively  meeting. 

Sub j  ects : 

Implementation 
Participation 
Management  Commitment 


Date:  04  Jan  90 


Summary  of  meeting: 

The  team  discussed  the  final  problem  areas  of  when  the 
submittal  arrives.  More  involvement  in  the  review  process  by 
the  COE  area  office  was  suggested.  The  discussion  moved  to 
the  distribution  procedures.  Solutions  were  proposed  to 
the  problems  previously  identified  in  this  step.  At  this 
point  the  discussion  got  off  subject  and  onto  the  0&M 
In-house  procedures. 

Notes : 

The  team  continued  to  struggle  with  an  incomplete  list  of 
problems  from  the  earlier  meetings.  Perhaps  it  is  Just  that 
at  the  earlier  time  the  team  was  not  warmed  up  and  was  not 
thinking  analytically  enough.  Time  was  spent  again  in  this 
meeting  discovering  new  problems,  when  the  team  had 
previously  decided  that  the  problem  list  was  complete  and 
moved  onto  solutions.  Perhaps  the  scope  of  this  first 
process  for  the  team  was  too  large  and  complex.  The  team 
perhaps  was  not  prepared  for  the  need  to  examine  their  work 
in  detail  unlike  they  were  used  to. 

This  was  the  first  meeting  back  after  a  four  week  layoff  due 
to  the  holidays.  The  effect  of  this  layoff  was  that  the 
team  was  slow  to  recall  where  they  had  gotten  to  before. 

They  were  somewhat  slow  too  in  regaining  their  momentum  and 
analytical  thinking.  But  by  the  second  half  of  the  meeting 
they  were  in  full  swing  again.  A  better  review  of  past 
accomplishments  and  an  agenda  for  the  current  session  by  the 
facilitator  might  have  speeded  this  recovery. 

Subjects : 

Management  Direction  -  Selection  of  process 

Implementation 

Fact  1 itator 

Participation 
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Date:  11  Jan  90 


Summary  of  meeting: 

The  2750  Air  Base  Wing  quality  day  was  announced  as  29  Jan. 
The  team  was  invited  to  a  photo  session  in  preparation  for 
that  event . 

The  team  brainstormed  solutions  to  the  COE  problems.  The 
steps  addressed  were  the  collect  step  through  the  send  to  HQ 
step . 

A  plan  to  invite  a  2750  ABW/COMM  representative  to  the  next 
meeting  to  discuss  ‘How  to  justify/request  the  assignment  of 
a  communications  engineer  to  WPAFB ‘  was  made. 

Notes : 

The  supervisor  of  the  engineering  branch  has  been  directly 
involved  in  the  process  action  team  meetings,  where  the 
remaining  participants  are  all  subordinate  to  this 
supervisor.  The  supervisor  has  been  open-minded  and  non- 
judgmental.  This  has  seemed  to  encourage  the  other 
participants  to  speak  out  freely  about  problems  they  face 
and  difficulties  they  have  in  solving  them.  They  appear 
eager  to  tell  the  supervisor  things  he  does  not  know  about 
how  bad  the  situation  often  is.  He  in  turn  frequently 
expresses  disbelief  at  what  his  people  have  had  to  put  up 
with.  This  encourages  the  other  participants  to  speak  out 
even  more  frankly,  seeing  that  their  comments  are  not 
falling  on  deaf  ears. 

My  observation  is  that  the  supervisors  participation 
regularly  in  the  team’s  work  has  been  positive.  Information 
has  been  brought  out  in  the  group  meetings  which  may  never 
have  been  revealed  had  not  the  atmosphere  of  interest,  and 
willingness  to  make  the  changes  proposed,  been  established 
by  the  supervisor. 

Sub j ects : 

Management  Direction  -  Selection  of  team 
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Date:  18  Jan  90 


Summary  of  meeting: 

The  team  arranged  for  two  representatives  of  the  area  Corps 
of  Engineers  office  to  attend.  Questions  were  allowed  both 
ways  to  better  understand  each  others  work.  Problems  that 
the  team  had  previously  identified  which  dealt  with  the 
interface  with  COE  were  referred  to  and  discussed. 

Notes : 

The  COE  representatives  were  visibly  impressed  with  the 
teams  work  and  open-minded  attitude.  They  were  put  on  the 
spot  to  answer  pointed  questions.  They  too  were  unsure  how 
the  base  used  some  of  their  input.  The  result  was  that  a 
greater  understanding  of  each  others  problems  and  reasons 
for  doing  things  was  gained.  In  addition,  mutual  respect 
was  increased  by  seeing  the  competence  of  "the  other  guys'. 
Several  agreements  were  reached  to  establish  better 
communication  and  to  make  changes  within  the  authority  level 
of  those  present  to  improve  the  interface. 

This  was  an  important  expansion  of  the  process  framework  by 
including  the  agency  who  is  the  upstream  provider  of  the 
design  submittals  as  well  as  the  downstream  customer  for  the 
base’s  output  of  comments.  A  new  appreciation  of  the  scope 
of  the  process  was  attained.  It  could  not  have  been  done 
without  management  inviting  the  representatives. 

Subjects : 

Process  Extension 
Implementation 
Management  Support 
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Date:  25  Jan  90 


Summary  of  meeting: 

Solutions  were  proposed  to  problems  identified  earlier  in 
the  90%  complete  step  of  the  COE  process.  The  solution 
phase  was  completed  with  this  step. 

The  team  agreed  to  proceed  now  into  the  actions  to  implement 
the  solutions,  and  who  would  be  responsible  for  each  action. 
Since  management  was  present  in  the  meeting  (one  member  was 
the  supervisor  for  the  COE  process)  no  further  approval  by 
management  was  needed.  Actions  were  detailed  for  the 
solutions  in  the  first  step  of  the  process,  designer 
selected . 

Some  of  the  implementation  actions  took  the  simple  form  of  a 
letter  to  an  outside  agency  describing  the  position  of  the 
Engineering  Branch  on  the  issues. 

Notes : 

The  team  was  relieved  and  encouraged  to  reach  this  milestone 
in  the  problem  solving  routine.  They  had  been  nearly  two 
months  on  the  solution  phase.  Finally  it  was  starting  to 
come  together.  The  team  chose  not  to  address  the 
measurement  techniques  for  the  actions  at  this  time. 

Getting  the  actions  detailed  and  assigned  was  enough. 
Measurement  would  follow. 

The  sometimes  fuzzy  ideas  which  had  been  the  solutions  began 
to  make  much  more  sense  when  converted  into  actions  for 
implementation . 

Subj  ects : 

Management  Support  -  Implementation 

Measurement 

Implementation 

Participation 


Date:  01  Feb  90 


Summary  of  meeting: 

The  contents  of  the  letters  to  the  COE  describing  the 
Engineering  Branch’s  position  on  issues  were  reviewed. 
Implementation  plans  for  the  submittal  arrives  step  were 
begun.  A  member  of  the  team  was  assigned  the  task  of 
establishing  a  plan  room  for  review.  This  was  an 
action  considered  to  be  worthy  of  immediate  implementation. 

Notes : 

The  team  was  thrilled  to  recommend  the  plan  room  for 
immediate  implementation.  It  was  a  concrete  example  that 
their  efforts  did  make  a  difference. 

The  primary  facilitator  for  the  process  action  team  observed 
in  this  research  had  no  formal  training  as  a  facilitator.  A 
trained  facilitator,  a  member  of  the  industrial  engineering 
branch,  attended  most  of  the  meetings  in  an  advisory 
capacity.  This  person  answered  questions  which  were  raised 
about  techniques  and  procedures  for  the  team’s  operations. 
However,  the  conduct  of  the  team  meetings  themselves  were 
left  to  the  untrained  primary  facilitator.  The  purpose  of 
this  assignment  was  to  afford  experience  for  this  individual 
to  "learn  on  the  Job"  how  to  facilitate  a  group. 

My  observation  is  that  this  did  not  seriously  injure  the 
ability  of  the  group  to  operate  effectively.  The  group 
consisted  of  technically  and  scientifically  trained  people, 
aware  of  problem  solving  techniques.  They  did  not  appear  to 
need  much  guidance. 

On  the  other  hand,  the  format  of  the  meetings  was  quite 
predictable  and  routine.  Could  a  trained  facilitator  have 
provided  variety  and  innovative  methods  for  generating 
interaction  and  information?  Would  such  methods  have  led  to 
better  performance  by  the  group?  Or,  were  the  results 
accomplished  as  good  as  any  which  may  have  been  achieved 
through  other  techniques? 

Sub j  ects : 

Faci 1 i tators 

Implementation 

Participation 
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Date:  08  Feb  90 


Summary  of  meeting: 

The  subject  of  the  plan  room  location  was  discussed.  The 
team  decided  to  put  the  decision  on  hold  until  management  of 
the  Engineering  Branch  can  gather  more  information. 

The  team  was  informed  of  a  memo  from  a  staff  engineer  from 
the  Operations  and  Maintenance  Branch  regarding  their  review 
process . 

Actions  to  implement  solutions  from  the  submittal  arrives 
step  were  determined  and  assigned. 

Notes : 

The  team  is  not  afraid  to  assign  implementation  actions  to 
other  responsible  individuals  within  the  organization. 

While  many  of  the  actions  are  accepted  by  team  members, 
others  are  assigned  out.  Having  the  management  supervisor 
present  assures  that  these  items  will  not  be  forgotten. 

Another  important  milestone  was  realized  by  further 
extending  the  process  to  the  Operations  and  Maintenance 
customer  interface.  By  receiving  the  memo,  the  team  came  to 
a  greater  understanding  of  their  process.  This  also  showed 
that  the  team’s  work  was  having  an  effect  throughout  the 
organization;  others  were  feeling  inclined  to  become  a  part. 
The  team  recognized  the  time  had  come  to  start  publicizing 
some  of  their  accomplishments. 

Sub j ects : 

Management  Support  -  Implementation 

Culture  Change 

Implementation 
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Date:  15  Feb  90 


Summary  of  meeting: 

Actions  to  implement  solutions  for  the  distribute  step  in 
the  COE  process  were  detailed.  In  particular,  a  new 
procedure  for  managing  COE  designs  was  beginning  to  emerge. 
The  team  detailed  these  new  steps  and  realized  they  were 
creating  a  new  position  description  for  a  person  to  manage 
Corps  of  Engineers  projects. 

A  guest  attended  the  team  meeting  for  the  third  week  in  a 
row.  The  guest  was  a  member  of  one  of  the  sections  in  the 
Engineering  Branch. 

Notes ■ 

The  maturity  and  flexibility  of  the  team  was  demonstrated  at 
this  point  by  their  acceptance  of  a  new  attendee  at  their 
meetings.  No  one  appeared  to  feel  threatened,  and  the 
guest’s  input  was  listened  to  and  accepted. 

This  guest  came  uninvited,  apparently  hearing  of  the 
accomplishments  of  the  team  and  wanting  to  be  a  part.  This 
is  testimony  of  the  interest  the  team  was  beginning  to 
develop  by  this  time. 

Many  of  the  actions  detailed  for  implementation  covered  a 
variety  of  the  solution  ideas  which  were  brainstormed 
earlier  in  the  problem  solving  routine.  The  linkage  of 
these  actions  with  the  solutions  they  implemented  was  kept, 
so  that  a  statement  could  be  made  to  management  regarding 
what  actions  implemented  which  solutions  and  ultimately 
which  problems  were  the  cause. 

Sub J  ects : 

Participation 
Culture  Change 

Measurement  -  Sell  Management 
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Date:  22  Feb  90 


Summary  of  meeting: 

The  detailing  of  actions  to  implement  solutions  to  problems 
in  the  COE  process  was  completed,  with  the  steps  of  collect 
through  90%  review  being  discussed. 

Notes : 

At  times  the  feathers  started  flying  as  implementation 
actions  proposed  hit  close  to  home  and  the  members  present 
tried  to  justify  their  past  way  of  doing  things  and  began  to 
wonder  what  the  actions  proposed  might  require  and  how  well 
they  might  work.  A  peacekeeping  role  by  the  facilitator  was 
required  to  soothe  these  worries  and  get  the  meeting  back  on 
track  and  off  the  arguments. 

Everyone  breathed  a  sigh  of  relief  as  the  implementation 
action  phase  of  the  problem  solving  routine  was  completed. 
And  with  it  was  an  appreciation  for  all  that  had  been 
accompl ished . 

Sub j  ects : 

Participation 
Faci 1 i tator 
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Date:  01  Map  90 


Summary  of  meeting: 

A  summary  list  of  the  processes,  problems,  solutions,  and 
implementation  actions  was  distributed.  The  list  was 
reviewed.  The  list  was  evaluated  in  light  of  a  recent 
announcement  that  with  the  breakup  of  the  AFRCE  function,  a 
new  COE  Milcon  Execution  Plan  will  be  developed.  This  means 
some  of  the  exact  implementation  actions  are  no  longer 
useful  because  they  were  designed  to  change  the  old  system. 
However,  it  was  pointed  out  that  the  work  of  the  team  will 
make  a  fine  foundation  to  the  preparation  of  the  new 
Execution  plan. 

Notes : 

Instead  of  changing  the  old  system,  the  work  of  the  team 
could  now  be  used  to  build  a  new  system.  This  was  something 
of  a  mixed  blessing.  Having  just  created  their 
implementation  actions,  the  team  would  have  been  encouraged 
to  see  them  put  into  effect  and  make  the  changes.  So  there 
was  some  disappointment.  However,  as  the  weight  of  the 
opportunity  to  build  the  new  system  from  scratch  sunk  in, 
the  team  could  recognize  the  significant  steps  already  taken 
to  make  sure  the  new  plan  worked  better  than  the  old. 

Sub j  ects : 

Management  Direction 
Participation 
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Date:  08  Map  90 


Summary  of  meeting: 

The  team  brainstormed  measures  of  success  for  the  actions 
they  had  previously  proposed.  Individual  action  measures 
were  determined,  and  how  to  gather  the  information 
discussed.  In  addition,  an  overall  goal  measure  of  reducing 
change  orders  was  proposed  as  the  bottom  line  gauge  of 
success . 

Notes : 

The  overall  goal  and  measure  of  success  came  up  here,  at  the 
end  of  the  problem  solving  routine.  It  might  have  been  more 
effective  to  have  been  established  from  the  beginning,  so 
that  the  team  would  have  been  constantly  aiming  at  it. 

With  the  measures  established,  and  the  responsibility  for 
implementations  assigned  in  the  previous  phase,  the  problem 
solving  routine  was  complete  for  the  Corps  of  Engineers 
design  review  process. 

Subjects : 

Measurement 
Implementation 
Goal  of  Improvement 
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Appendix  E. 


Process  Flow  Diagrams 


Corps  of  Engineers 


Designer  Selected 
Contract  Set 


Submittal  Arrives 
(107.)  ,307.,  (60%)  ,907. 


Distribute  Per  List 


Collect,  Compile,  Filter  Comments 


Send  to  Headquarters 
Design  Review  Meeting 


Repeat  from  'Submittal  Arrives'  Until 
90%  Submittal  Level  Is  Reached 


End 


Figure  35.  Corps  of  Engineers  Design  Review  Process 
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0  &  M  Archi tect/Eng ineer 


Designer  Selected 
Contract  Set 


Assign  Task 


Submittal  Arrives 
10*/.  ,35%,  907. ,  100% 


Distribute  Per  List 


Collect,  Compile,  Filter  Comments 


Send  to  Architect/Engineer 
Design  Review  Meeting 


Repeat  from  "Submittal  Arrives'  Until 
100%  Submittal  Level  Is  Reached 


90%  Pre-final  Review  Distribution  by  DEEE 


I 

End 


Figure  36.  0  &  M  Archi tect/Engineer  Design  Review  Process 
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0  &  M  In-House 


Designer  or  Team  Selected 


Prepare  35%  Cost  Estimate 
Compare  to  Programmed  Amount 


If  Cost  Unacceptable,  Reprogram 


90%  Pre-final  Review  Distribution  by  DEEE 


Compile  Comments 


Forward  to  Engineer  to  Use  or  Respond  To 


End 


Figure  37.  0  &  M  In-house  Design  Review  Process 
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ASD  Archi tect/Engineer 


Submittal  Arrives 


Distribute  Per  List 


Collect,  Compile 


Filter  Comments 


Send  to  ASD 


Repeat  from  "Submittal  Arrives"  Until 
95 7.  Submittal  Level  Is  Reached 


Receive  AFLC  95/C  Comments 


Receive  Complete  Documents  for  Contracting 


Figure  38.  ASD  Architect/Engineer  Design  Review  Process 
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Aggend i x  Design  Review  Process 

Problems Solutions^.  Act Ioqs A  §Qd  MeaSUESS 

This  appendix  contains  a  summary  of  the  brainstorming 
sessions  of  the  performance  improvement  team  as  they  studied 
the  design  review  process.  The  team  first  identified  the 
problems  in  each  step  of  the  Corps  of  Engineers  design 
review  process.  They  then  proposed  solutions  to  these 
problems.  Finally,  actions  were  recommended  to  implement 
these  solutions  and  measures  of  success  proposed. 

The  steps  in  the  Corps  of  Engineers  design  review  process, 
together  with  three  other  processes  the  team  identified,  are 
outlined  in  Appendix  E.  For  ease  of  reference,  the 
problems,  solutions,  actions,  and  measures  are  consolidated 
for  each  step  of  the  process.  This  allows  the  reader  to  see 
how  the  problems  in  a  step  were  solved.  However,  the 
procedure  the  team  followed  to  generate  this  information  was 
not  as  shown.  Rather,  the  team  first  identified  problems  in 
all  of  the  Corps  of  Engineers  process  steps  together.  Next 
solutions  were  proposed  to  all  the  problems  in  the  entire 
process.  Actions  were  then  detailed  to  implement  the 
solutions.  Finally,  measures  were  proposed  to  track  the 
actions . 
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As  of :  14  Mar  90 


DESIGN  REVIEW  PROCESS  ” PAT ' 

Corp-of-Engineer  (COE)  Design  Review 
I.  Designer  Selection  Step 

A.  Problem  Areas 

1.  Since  no  input  requested  from  base,  get  stuck 
with  A-E  (sometimes  bad). 

2.  AE  selected  from  Cincinnati/Kentucky,  too  far. 

B.  Solutions 

1 .  Input  from  ba3e 

a.  Get  someone  on  committee  (TDY) . 

b.  Get  to  review  candidates  or  make 
recommendations . 

c.  Provide  comments  from  previous  AE  designs. 

d. .  Learn  the  COE  selections  process. 

e.  If  COE  selection  procedure  includes  location 
factor,  use  WPAFB  as  optimal  location,  not 
Louisville.  If  not,  have  them  include  it. 

f.  Contract  requires  ample  site  visits  by  A-E 
for  predesign  verification. 


C.  Implementation  Actions 

1.  In  letter  1  to  HQ  AFLC  with  a  copy  to  COE: 

a.  Request  support  for  a  base  representative  to 
attend  final  selection  committee  and  participate 
as  a  voting  member.  Member  will  communicate 
base  evaluation  of  A-E’s  to  selection  committee. 

b.  Request  that  COE  be  instructed  to  notify 
base  of  top  candidates  from  A-E  pre  selection 
committee  when  known. 

2.  Include  in  01  for  DEEED  project  mgrs . : 

a.  To  bring  base  evaluation  of  A-E’s  to  final 
selection  committee  for  briefing. 
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b.  When  design  instruction  is  issued,  project 
manager  will  send  letter  3  to  COE  and  all 
reviewing  agencies,  informing  of  base  CE  POC . 

NOTE:  Above  actions  superceded  by  the  MILCON 

Project  Management  Plan  (MPMP) .  WPAFB  will  correct 
this  problem  area  by  participating  in  the 
development  and  implementation  of  the  MPMP. 

OPR:  DEEP  ECD :  Set  by  MAJCOM 


3.  Budget  funds  for  project  managers  to  attend 
final  selection  committee. 

OPR:  DEE  ECD:  Each  01  Oct 

D.  Measuring  Progress/Success. 

1.  Percent  of  Final  Selection  Committee  meetings 
attended  each  FY  with  a  goal  of  100%. 

OPR:  DEEED  for  tracking  goal  percent  ECD:  Ongoing 

II.  Submittal  Arrives  Process  Step: 

A.  Problem  Areas 

1.  Routed  through  COE,  time  is  lost. 

2.  Not  enough  copies. 

3.  Don’t  see  project  after  95%  comments. 

4.  Submittal  levels  may  not  be  appropriate  (ie:  10% 
for  scope,  90%  for  tech  review,  are  30%  and  60% 
needed) 

5.  Too  many  separate  voices  giving  direction  to  AE. 

6.  AE  relies  on  us  for  QC  and  to  catch  mistakes 
which  slows  down  submittal  timing  requirements. 

7.  COE  const,  representative  on-base  is  not  active 
enough  in  design  review. 

B.  Solutions 

1.  Include  survey  notes  in  early  submittal,  and 
other  predesign  investigation  info  (soil  logs, 
utilities) 
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2.  In  lieu  of  current  35%  submittal,  request  a  10% 
submittal  to  include  such  as  structural,  electrical, 
mechanical  systems,  suggested  floor  plan,  and  site 
plan,  and  site  investigation  results. 

3.  Changed  to  20%,  not  at  50%.  Clearly  outline 
what’s  required  to  get  uniformity.  10%  could  be 
presentation  with  alternatives. 

4.  Get  more  dollars  released  for  1st  submittal. 

5.  60%  added  by  COE  as  needed  for  large  projects. 

6.  10,  35,  90%  submittals  are  needed  and  need  to  be 
well  defined  (checklist). 

7.  100%  needed  before  contracting  starts  to  make 
last  check  of  comments. 

8.  Number  of  copies/route  through  COE. 

9.  Decide  how  many  needed  and  get  into  DI  (5) . 

10.  Establish  consistent  distribution  channel. 

11.  Locate  plan  review  room  that  is  a  consistent 
place,  that  can  be  reserved  for  1-2  days.  Everyone 
has  their  own  place  now. 

12.  Be  informed  of  overall  design  schedule.  Have  it 
included  in  submittal  from  A-E. 

13.  For  100%  submittal,  get  on  distribution  list 
with  time  to  comment  before  contract  start. 

14.  Have  drawing  control  log  on  cover  sheet  of 
drawings  showing- who  should  get  review  sets. 

15.  Since  too  many  voices,  minimize  list  of  people 
who  see  submittals  to  review.  All  comments  should 
go  through  COE  project  manager  as  lone  voice  to  A-E. 

16.  Need  opportunity  to  discuss  and  defend  comments, 
prior  to  project  manager  and  deleting. 

17.  Instruct  A-E  to  go  back  and  arbitrate  deleted 
comments . 

18.  Have  mandatory  design  review  meeting  where 
deleted  comments  are  identified  and  discussed. 
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19.  A-E  using  AF  for  QC 

a.  Get  tough  with  A-E. 

Hold  responsible  for  their  mistakes. 

b.  Return  inferior  submittals  without  review  and 
demand  reworked  submittal. 

o.  Withhold  payment  for  inferior  submittals. 

d.  Make  sure  A-E  is  given  adequate  time  to 
design,  so  they  have  time  to  do  job  right. 

20.  COE  design  rep  on  base. 

a.  COE  develop  position  (deputy?)  to  be 
responsible  for  design  review  on  base. 

b.  Have  COE  design  engineer  assigned  to  base 
for  project  duration  during  construction 
(especially  when  AFRCE  dissolved). 

C.  Implementations  Actions 

1.  Send  letter  2  to  COE,  copy  to  AFRCE 

a.  Detail  what  base  wants  to  see  in  submittals, 
particularly  evidence  of  site  investigation 
(survey  notes,  etc.)  in  early  submittals.  Send 
submittal  requirement  description  from  O&M 

pro j  ects . 

b.  Emphasize  to  COE  need  extensive  early 

design  work. 

c.  Inform  COE  of  new  procedure  for  base  to  send 
letter  3. 

d.  Ask  that  COE  have  design  schedule  included  in 
first  submittal  by  A-E. 

e.  Ask  that  COE  have  A-E  include  distribution 
list  for  review  sets  on  cover  sheet. 

f.  Ask  that  COE  send  copy  of  100%  submittal  to 
base  immediately  upon  receipt  from  A-E  for  base 
to  check  critical  items. 

2.  In  letter  3,  base  project  mgr  will  detail  what 
submittals  are  desired  for  the  project.  Include 
this  in  0 . I . 
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3.  Submit  work  requests  for  plan  room. 

OPR:  DEE  ECD :  01  Mar  90 

4.  Letter  3  is  to  include  an  invitation  to 
reviewing  agencies  such  as  user,  EM,  Comm,  DEF ,  DEM, 
Safety,  DEEC ,  etc.,  to  comment  to  DEEED  PM.  Include 
this  in  0 . I . 

OPR:  ECD: 

5.  Inform  COE  in  letter  2  that  base  needs  design 
review  meeting  on  each  project  for  chance  to  defend 
base  comment  which  HQ  AFLC  or  AFRCE  have  deleted,  at 
60/C  or  90/C  as  a  minimum  as  appropriate 

01 R:  ECD: 

6.  Include  in  DEEED  Project  Mgr  O.I.: 

a.  PM  to  keep  track  of  comments  made  from  base, 
and  resolve  conflicts  or  elevate  to  higher 
authority  as  needed  to  get  results. 

b.  PM  to  inspect  each  newly  received  submittal 
for  completeness  against  requirements  for 
submittal  level.  If  deficient,  a  letter  will  be 
sent  to  HQ  AFLC  complaining,  and  suggesting 
payment  be  withheld. 


NOTE:  Except  for  action  3,  all  of  the  above  actions 

will  be  satisfied  through  the  development  and 
implementation  of  the  MPMP.  Item  l.f.  can  be 
deleted . 

OPR:  ECD « 

7.  In  letter  2,  request  COE  assign  design  rep  to 
base  . 

NOTE:  Delete,  since  this  will  be  addressed  when  the 

AFRCE  i3  disbanded. 

III.  Distribute  Design  Documents/Drawings  Step 
A.  Problem  Areas 

1.  Are  the  right  people  on  the  list?  (ie:  make 
sure  COE  const,  staff  included). 

2.  User  gets  design  package  last. 
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Personnel  changes  at  user,  changes  scope. 
Local  COE  office  comments  only  at  90%. 


3  . 

4  . 


B.  Solutions 

1.  Create  design  review  position  in  DEEC  (Right 
people  on  list?) 

2.  Have  yearly  review  by  mgt  of  who  is  on  list  and 
update  accordingly. 

3.  Have  design  review  room  and  send  letters  to 
review  if  interested. 

4.  Include  DEE  and  DEEE  for  comments. 

5.  Develop  user  design  checklist  to  make  sure 
consideration  is  given  (CID,  disruption,  equipment 
by  user,  scope,  get  copies  from  A-E’s). 

6.  Add  checklist  to  user  review  letter. 

7.  To  compensate  for  user  personnel  changes  and 
scope  change,  establish  consistent  POC  at  other 
agencies  for  review. 

8.  Insure  programming/ 1 39 1  is  complete  and  up-to- 
date  . 

C.  Implementation  Actions 

1.  Add  DEE  and  DEEE  to  distribution  list  for 
review.  Combine  current  DEEE  and  DEEED  distribution 
lists.  Submit  revised  list  to  the  PAT  for  review. 

OPR:  DEEE  ECD :  01  Apr  90 

2.  Add  to  DEEE  0.1.  and  DEEED  O.I.: 

a.  All  project  review  to  be  distributed  from  and 
monitored  by  DEEE.  DEEED  will  pass  submittals 
to  DEEE  to  place  in  plan  room  and  send  letters 
to  reviewing  agencies.  DEEE  to  add  user  review 
checklist  to  letter  going  to  user. 

OPR:  DEEED  ECD:  15  Apr  90 
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3.  Develop  user  review  checklist,  to  prompt  them 
what  to  look  for,  scope,  phasing,  operations  and 
maintenance,  C.I.D.  For  at  least,  the  initial  go- 
around ,  request  that  users  provide  checklist 
evaluation/improvement  comments. 

OPR:  DEEED  ECD:  15  Apr  90 

4.  In  letter  3,  ask  all  reviewing  agencies  to 
identify  their  POC  and  alternate.  Attach  Form  1391 
for  project  to  letter  to  alert  reviewers  to  scope  of 
project.  Add  this  to  0.1.  for  DEEED  PM. 

NOTE:  Incorporate  implementation  action  4  above  in 

MPMP . 

D.  Measuring  Progress 

1.  For  implementation  actions  C.2  and  C.3  above, 
evaluate  checklist  comments  provided  by  users. 
Incorporate  improvements  as  feasible. 

IV.  Collect,  Compile,  Filter  Step 

A.  Problem  Areas 

1.  Sometimes  send  comments  after  suspense. 

2.  Some  base  comments  deleted  by  HQ  with  no 
feedback  or  explanation. 

3.  No  DEEC  reviewer  assigned;  inspector  not 
assigned  until  contract  start. 

4.  COMM  organization  comments  normally  late. 

5.  COMM  engineering  at  Griffis  (485  EIG)  not  giving 
adequate  support,  so  wiring/splicing  not  getting 
into  contract. 

6.  DEM  review  of  elec,  HVAC ,  etc.  not  getting  to 
staff  engineers  (focal  point). 

B.  Solutions 

1.  Since  base  comments  deleted  at  HQ,  make  a  ruling 
that  comments  cannot  be  rejected  w/o  contacting 

c  o  mme  n  t  o  r  . 

2.  Ensure  that  every  COE  review  has  review  meeting 
where  deleted  comments  can  be  defended. 
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3.  Since  COMM  design  occurs  at  Griff iss  AFB ,  -  need 
to  get  Griffiss  involved  in  timely  manner. 

4.  Get  COMM  engineer  assigned  to  base  COMM,  or  get 
COMM  engineer  in  DEE  to  preclude  late 
actions/comments . 

5.  Since  comments  sent  after  suspense,  need  a 
design  review  meeting  and  review  room/area. 

6.  To  get  DEEC  inspector /reviewer  involved  from  the 
start,  send  design  to  DEEC  for  review  of  each  step. 

7.  Assign  DEEC  person  to  project  for  review  and 
follow  on  construction  at  start  of  the  project. 

8.  For  DEM  review,  send  directly  to  staff 
engineers . 

9.  Make  sure  staff  engineers  are  coordinating  with 
shops  for  comments  (suggest  they  have  review  meeting 
with  shop  personnel). 

10.  For  EM  review  -  understand  and  communicate  their 
organization  and  develop  workable  procedure  to  get 
review. 

Implementations  Actions 

1.  Send  letter  4  to  2046  Comm:  Current  system  isn’t 
working.  Suggest  getting,  Comm  design  engineer 
assigned  to  base.  Alternate  plan  to  get  Comm  engr . 
in  base  DE  for  design. 

NOTE:  Include  in  letter  that  the  2046  needs  to 
respond  to  decentralization  resulting  from  the 
implementation  of  the  MFMF .  Now  the  base  has  an 
even  greater  responsibility. 

OPR:  DEEC  ECD :  Subject  to  FY92  MCP 

2.  Make  sure  review  letters  to  DEM  are  routed  to 
Staff  Engineer  Office. 

OPR:  DEEE  ECD:  01  Apr  90 

3.  Send  memo  to  DEM,  informing  them  that  all  future 
reviews  will  come  to  staff  engineers.  Suggest  that 
they  have  review  meeting  with  shops.  They  may 
invite  DEEED  representative  to  attend  if  desired. 


OPR:  DEEED 


ECD:  01  Mar  90 


4.  Obtain  new  wing  and  base  CE  organizational 
charts,  including  EM  under  DE ,  and  phone  lists  for 
EM. 

OPR:  DEEE  ECD :  Done 

5.  Set  up  session  between  all  EM  branch  chiefs  and 
DEE  section  chiefs  to  formalize  processes  for 
permitting,  etc.  Show  them  flow  chart  of  review 
processes  developed  in  PAT. 

OPR:  DEE  ECD:  15  Mar  90 

D.  Measuring  Progress 

1.  For  implementation  action  C.2  above,  evaluate 
process  at  the  end  of  FY  90  to  determine  if  review 
of  the  design  package  by  the  Staff  Engineer's  Office 
has  improved  and  that  shop  comments  are  included  in 
the  final  review.  Also,  conduct  a  random  survey  of 
the  shops  to  determine  if  they  has  an  opportunity  to 
review  the  design  package  and  provide  comments. 

OPR:  Project  Manager  for  the  random  ECD:  Depends  on 

survey  and  DEEED  for  adding  FY92  MCP 

to  the  Design  Process  0.1. 

2.  For  C.5  above,  success  will  be  a  measure  of 
having  a  productive  meeting  with  EMO . 

V.  Design  Review  Meeting 

A.  Problem  Areas 

1.  HQ  filters  comments  without  meeting  with  base 
before  HQs  sends  comments  to  COE/AFRCE. 

2.  Do  not  have  design  review  meetings  on  every 
project . 

3.  Do  not  conduct  scope  focused  review  meetings  at 
10%  or  30%. 

4.  COMM  representatives  are  only  good  for  technical 
questions.  Have  no  authority. 

5.  DEEC  not  notified. 

6.  Design  rep.  and  construction  rep  not  identified 
in  MCP  report. 
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B.  Solutions 


1 .  Have  HQ  instruct  COE  to  include  in  contract 
provision  for  all  meetings  at  WPAFB. 

C.  Implementation  Actions 

1.  In  letter  1  to  HQ  AFLC ,  request  all  design 
review  meetings  be  conducted  at  WPAFB. 

OPR:  ECD : 

NOTE:  Include  in  MPMP  development  and 

implementation. 

VI.  Scope  Review  at  90% 

A.  Problem  Areas 

1.  Never  see  100%  for  checking  comments. 

2.  Since  no  continuity  in  review  among  user  &  COMM, 
different  folks  looking  at  each  submittal,  or 
command/personnel  changes. 

B.  Solutions 

1.  Make  sure  user,  AFRCE ,  A-E,  DEEP  are  involved  at 
10%  or  30%. 

2.  Identify  DEEC  rep  on  MCP  Report  from  outset. 

3.  Discussed  100%  design  review  with  COE  during 
18  Jan  1990  meeting  at  Wr ight-Patterson  AFB .  They 
do  not  send  out  for  new  review,  unless  the  project 
sits  on  the  shelf;  then  sent  to  the  AFRCE. 

Therefore,  have  the  COE  send  the  100%  for  a  Base 
check  for  each  project. 

4.  Need  Base  Comm. /user  continuity  when  conducting 
design  review. 

a.  Assign  a  single  point  of  contact  (POC)  within 
these  organizations.  Have  POC  attend  design 
review  meetings. 

b.  When  the  Design  Instruction  is  issued,  the 
project  manager  should  establish  the  POCs . 

c.  Make  certain  that  each  organization  has  an 
alternate/backup;  someone  who  can  continue  the 
process  if  the  primary  is  unavailable. 


318 


» 


C.  Implementation  Actions 

1.  Include  DEEC  rep  and  DEEED  rep  for  each  COE 
project  on  MCP  report,  all  reports. 

OPR:  DEEP  ECD:  20  Mar  90 

2.  In  letter  1  to  HQ  AFLC ,  request  that  COE  not  be 
permitted  to  send  projects  out  to  bid  if  they 
haven’t  been  reviewed  by  the  base  in  the  last  6 
months . 

3.  In  letter  4  to  2046  Comm,  inform  them  of  the  new 
review  procedure  adopted,  and  that  at  the  outset  of 
each  project  they  will  be  asked  for  a  primary  and 
alternate  P.O.C. 


NOTE:  Include  actions  2  and  3  in  the  development  of 

the  MPMP . 

OPR:  ECD:  Contingent  on  FY92  MCP 

D.  Measuring  Progress 

1.  For  C.l  above,  action  will  be  considered 
successful  when  the  next  MCP  Report  is  issued. 

E.  Overall  Measure  of  Progress/Suecess 
1.  Minimize  Change  Orders 

a.  Establish  a  baseline  for  the  last  five  (5) 
years,  FY85-89  to  include  by  project  change 
orders  required  by  (1)  design  errors  (2)  design 
omissions  (3)  mission/user  changes  and  (4) 
latent  conditions. 

b.  Measure  change  orders  as  a  percentage  of  the 
contract  price. 

c.  Goal  is  to  reduce  change  orders  caused  by 
(1)  and  (2)  to  zero  (0)  and  reduce  (4)  by 
improved  site  investigation. 
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Items  From  Visit  By  Corp-of -Engineers  (COE)  Reps 


1)  Vehicle  exists  to  include  base  input  in  selection,  now 
evidently  just  getting  to  AFRCE. 

2)  COE  to  BCE  communication  gets  lost  at  AFRCE.  Base 
doesn’t  hear  about  selection/pre-design  meetings.  Have  COE 
contact  base  directly. 

3)  Add  discussion  of  upcoming  project  status  to  monthly  COE 
meeting  at  base.  Have  base  design  rep.  attend. 

4)  Establish  procedure  design  support  unit  and  COE  for 
early  coordination  on  D.I.’s,  selection  and  pre-design. 

5)  Get  copy  of  COE  book  detailing  what  each  submittal  is 
composed  of . 

6)  AF  can  choose  what  submittals  it  wants  (10%,  60%  are 
extra)  . 

7)  Can’t  change  35%  requirements,  except  maybe  on  a 
specific  item. 

8)  A-E  only  held  responsible  for  design  errors  if:  1)  No 

damage  to  gov’t  2)  Gov’t  didn’t  mislead  A-E  with  incorrect 
dwgs.  A-E  is  responsible  for  damages  and  to  redesign  the 
solution.  Performance  reflects  on  ACASS  evaluation  rating. 

9)  A-E  not  payed  for  extra  submittals  when  caused  by  A-E’s 
poor  work . 

10)  Base  COE  members  have  various  design  duties,  but  no 
specific  member  assigned  exclusively  for  design  problems. 

11)  100%  is  checked  internally  and  constructability  review 
made  before  RTA  and  put  on  street. 

12)  COE  issues  amendments  during  bidding  to  answer 
questions,  so  BASE  can  get  final  comments  to  COE  for 
Inclusion  in  amendment. 

13)  See  if  base  can  tie  into  ACASS  network. 
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Appendix  G.  Quality  Questionnaire 


The  following  sections  will  ask  for  information  from  you 

about  your  organization.  Please  answer  each  question  to  the  best 
of  your  abilities  on  a  separate  sheet  of  paper.  For  example,  the 
primary  intention  for  the  following  section  on  leadership  is  to 
determine  if  your  organization's  leadership  emphasizes  quality 
as  part  of  the  company's  value  system,  through  both  personal 
action  and  through  demands  on  employees.  Use  the  following  scale 
to  rate  all  questions. 

1  -  Strongly  disagree  5  -  Slightly  agree 

2  -  Moderately  disagee  6  -  Moderately  agree 

3  -  Slightly  disagree  7  -  Strongly  agree 

4  -  Neither  agree  nor  disagree 


PREFACE:  The  intent  of  this  questionnaire  is  to  query 
individuals  who  have  been  intimately  involved  with  the  CE 
PATs  and  those  who  have  not,  then  to  note  significant  differences. 
Please  indicate  on  your  separate  answer  sheet  if  you  have  or 
have  not  been  intimately  involved  with  the  PATs  in  Civil 
Engineering.  Any  information  regarding  your  experiences  in 
your  organization's  quality  movement,  if  any,  would  be 
appreciated. 


I .  LEADERSHIP 

A.  Supervisory  Communication 

1.  Your  supervisor  encourages  you  to  let  him/her  know  when 
things  go  wrong  on  the  job. 

2.  The  communication  between  you  and  your  supervisor  is  good. 

3.  You  are  free  to  tell  your  supervisor  that  you  disagree 
with  him/her. 

4.  Work  center  problems  are  often  discussed  as  a  group,  with 
supervisors  and  workers  openly  and  honestly  discussing 
the  issues. 

5.  Your  supervisor  is  fully  aware  of  work  center  problems. 

B.  Participative  Decision  Making 

1.  This  organization  is  always  moving  toward  the  develop¬ 
ment  of  new  answers. 


321 


2.  In  your  organization,  people  are  allowed  to  try  to  solve 
the  same  problem  in  different  ways. 

3.  Creativity  is  encouraged  in  your  organization. 

4.  People  in  your  organization  are  always  searching  for 
^esh.  new  ways  of  looking  at  problems. 

5.  The  leadership  acts  as  if  people  in  your  organization 
are  creative. 

C.  Commitment  to  Quality 

1.  Quality  is  more  than  just  the  latest  fashionable  "buzzword". 

2.  Your  boss  is  sincerely  interested  in  giving  you  time  to 
do  the  job  right. 

3.  Product  defects  are  an  unwanted,  but  inevitable  by¬ 
product  of  deadlines  and  schedules. 

4.  Your  supervisor  tries  hard  to  remove  restrictions  that 
limit  performance. 

D.  Shared  Vision 

1.  Continually  improving  work  results  are  a  realistic 
goal  . 

2.  Our  organization  continually  works  to  improve  overall  end 
results. 

3.  When  we  work  to  increase  quality,  we  don’t  necessarily 
decrease  productivity. 


II.  STRATEGIC  QUALITY  PLANNING 
A.  Goal  Clarity 

1.  You  know  exactly  what  is  expected  of  you  in  performing 
your  job. 

2.  You  understand  the  priorities  associated  with  what  you 
are  expected  to  accomplish  on  the  job. 

3.  Your  supervisor  clearly  indentifies  those  work  processes 
that  need  improvement. 


B.  Goal  Congruence 


1.  Your  organization's  goals  make  a  lot  of  sense. 

2.  You  have  a  personal  stake  in  your  organization's 
effectiveness . 

III.  HUMAN  RESOURCE  MANAGEMENT 

A.  Training  Adequacy 

1.  You  have  all  the  skills  you  need  in  order  to  do  your 
job. 

2.  You  have  been  provided  enough  training  to  ac^uir*.  the  necessary 
skills  to  do  your  job  well. 

B.  Involvement 

1.  You  feel  personally  responsible  for  the  work  you  do 
on  your  job. 

2.  You  often  make  suggestions  for  improving  work  conditions 
and  processes. 

3.  Management  encourages,  and  often  discusses  with  the  work 
force  new  ideas  for  improving  how  jobs  are  done. 

C .  Empowerment 

1.  Rules  and  regulations  of  your  organization  are  not  meant  to 
hinder  your  performance. 

2.  Your  ideas  for  improving  work  conditions  and  processes 
are  often  implemented. 

3.  You  are  given  opportunities  to  provide  your  own  ideas 
to  try  to  improve  "the  way  things  are  done"  in  your 
organization. 

D.  Expectancy 

1.  Your  supervisor  consistently  rewards  top  performers. 

2.  The  people  who  most  deserve  recognition  receive  that 
recognition . 


323 


E.  Role  Clarity 


1.  You  know  exactly  what  is  expected  prior  to  undertaking 
any  specific  task. 

2.  You  know  who  makes  the  decisions  in  your  organization 

and  huw  the  dcCaS^oria  are  reached. 

F.  Recognition/Feedback 

1.  Your  least  frequent  feedback  is  criticism. 

2.  Your  supervisor  provides  immediate  feedback  when  work 
results  are  good. 

3.  You  usually  know  whether  or  not  your  work  is 
satisfactory. 


IV.  QUALITY  ASSURANCE  OF  PRODUCTS  AND  SERVICES 

A.  Quality  system  *■ 

1.  You  have  no  problem  obtaining  the  tools,  equipment  and 
supplies  necessary  to  do  your  job. 

2.  You  are  held  accountable  for  your  mistakes  and  are 
required  to  take  action  to  prevent  their  recurrence. 

3.  This  organization  attempts  to  solve  its  problems  as  best 
it  can. 

V.  QUALITY  RESULTS  V 

A.  External  Measures 

1.  Your  organization  is  as  good  as  any  other  similar 
organization. 

2.  Complaints  are  rarely  ever  received  about  the  work 
of  your  organization. 

3.  The  results  of  work  in  your  organization  meet  your 
customers  standards  . 

B.  Internal  Measures 

1.  Your  organization  is  the  best  it  has  ever  been. 

2.  In  your  organization  everyone  knows  how  important 

it  is  to  do  things  right  » 
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VI .  CUSTOMER  SATISFACTION 

A.  Responsiveness 

1.  Your  customers  have  the  right  to  call  and  talk  to  the 
person  who  did  the  work  if  they  are  unhappy  about  it. 

2.  If  a  customer  complains  about  something,  immediate 
action  is  taken  to  identify  the  problem. 

3.  Customer  satisfaction  is  the  whole  reason  we  work  for- 
the  phrase  "satisfying  our  customers"  receives  more  than 
just  "lip  service"  in  our  organization. 

4.  Customers  are  given  the  fastest  possible  feedback  to 
their  questions. 

5.  It  is  easy  for  the  customer  to  get  in  contact  with  the 
experts . 

6.  Customers  receive  courteous  treatment  from  your 
organization . 

B.  Feedback 

1 .  The  most  important  measures  of  your  performance  are 
obtained  through  customer  feedback. 

2.  You  always  receive  information  on  our  customers 
reactions  when  it  involves  your  work. 

3.  In  this  organization,  you  often  make  changes  based  on 
inputs  from  your  customers . 


Thank  you  for  your  help  in  this  endeavor. 

If  you  have  any  constructive  comments  regarding  improvement  of 
your  quality  program,  please  indicate  these  for  consideration 
by  your  organization. 
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6 

5 

6 

35 

6 

5 

6 

5 

36 

5 

3 

6 

6 

37 

7 

4 

6 

5 

III  . 

94 

71 

90 

79 

330 


Question 

Member 

L  Member 

2  Member 

3  Member  4 

38 

4 

1 

5 

5 

39 

6 

5 

7 

6 

40 

5 

5 

6 

6 

IV. 

15 

11 

18 

17 

41 

6 

6 

6 

6 

42 

7 

4 

6 

4 

43 

6 

3 

7 

5 

44 

5 

3 

4 

45 

7 

3 

7 

4 

V. 

26 

16 

26 

19 

46 

7 

2 

7 

6 

47 

7 

5 

6 

5 

48 

7 

1 

6 

5 

49 

6 

5 

7 

5 

50 

7 

3 

6 

5 

51 

7 

5 

7 

5 

52 

6 

6 

7 

5 

53 

5 

5 

6 

4 

54 

4 

5 

6 

4 

VI  . 

56 

37 

58 

44 

Total 

322 

219 

318 

275 

Question 

Control  Group 
Member  1  Member  2 

Post- 

Member 

Test 

3  Member  4 

1 

7 

6 

7 

7 

2 

5 

7 

7 

7 

3 

7 

6 

7 

7 

4 

6 

5 

7 

6 

5 

5 

5 

6 

6 

6 

6 

4 

4 

5 

7 

7 

5 

6 

6 

8 

7 

3 

6 

6 

9 

7 

5 

6 

5 

10 

6 

3 

6 

5 

1  1 

7 

2 

6 

4 

12 

7 

5 

7 

6 

13 

5 

7 

1 

4 

14 

6 

4 

6 

6 

15 

7 

5 

7 

5 

16 

6 

6 

7 

6 

17 

6 

3 

7 

6 

I  . 

95 

72 

96 

89 

18 

7 

5 

7 

6 

19 

6 

6 

7 

6 

20 

5 

4 

7 

6 

21 

6 

5 

7 

4 

22 

7 

5 

6 

6 

II  . 

31 

25 

34 

28 

23 

1 

4 

7 

3 

24 

1 

3 

7 

4 

25 

7 

6 

7 

7 

26 

5 

5 

6 

6 

27 

4 

4 

7 

6 

28 

7 

5 

7 

6 

29 

5 

5 

7 

6 

30 

6 

5 

7 

6 

31 

7 

5 

4 

6 

32 

7 

6 

4 

6 

33 

5 

6 

7 

6 

34 

4 

5 

7 

5 

35 

5 

5 

7 

5 

36 

5 

6 

7 

6 

37 

7 

6 

7 

6 

III  . 

76 

76 

98 

84 

332 


Question 

Member  1 

Member 

2  Member 

3  Member  4 

38 

5 

2 

6 

2 

39 

7 

5 

7 

6 

40 

7 

4 

7 

5 

IV. 

19 

11 

20 

13 

41 

7 

7 

7 

6 

42 

5 

5 

7 

3 

43 

4 

5 

7 

4 

44 

4 

5 

4 

6 

4  b 

5 

6 

7 

5 

V. 

21 

23 

28 

18 

46 

7 

6 

7 

6 

47 

7 

5 

7 

4 

48 

6 

5 

7 

3 

49 

7 

4 

7 

3 

50 

7 

4 

7 

5 

51 

7 

5 

7 

6 

52 

5 

3 

6 

3 

53 

6 

4 

6 

3 

54 

6 

3 

6 

3 

VI  . 

58  * 

39 

60 

36 

Total 

300 

246 

336 

268 

Appendix  1 . 


Comparison  of  Group  Pre-Test  Results 


Question 

Treatment 

Average 

Control 

Average 

t 

tcri  t 

Resul t 

1 

6.6 

6.25 

0.499991 

2.36 

same 

2 

6.8 

6 . 5 

0 . 696009 

2.36 

same 

3 

6 . 8 

6 . 5 

0 . 696009 

2.36 

same 

4 

5 . 2 

5 

0 . 196957 

2.36 

same 

5 

5 . 8 

6 

-0.40788 

2.36 

same 

6 

4 . 2 

5 . 5 

-1 .06016 

2.36 

same 

7 

5 . 6 

5.5 

0. 177993 

2.36 

same 

8 

4.8 

6 

-1 . 29582 

2.36 

same 

9 

4.8 

5 . 25 

-0 . 42917 

2.36 

same 

10 

3.8 

4.75 

-0 . 80244 

2.36 

same 

1 1 

4.6 

4 . 333333 

0 . 246765 

2.45 

same 

12 

6.2 

5.75 

0.724418 

2.36 

same 

13 

5.6 

4 

1  .  172197 

2.45 

same 

14 

6.4 

5 . 5 

1.6185 

2 . 36 

same 

15 

6 

5.75 

0.338948 

2 . 36 

same 

16 

4.8 

5 

-0. 17983 

2.36 

same 

17 

5.4 

6 

-0.73879 

2.36 

same 

I  . 

83.2 

85.25 

-0.22359 

2.36 

same 

18 

*  6.4 

6.5 

-0.30161 

2.36 

same 

19 

5.8 

6.5 

-1  .  10967 

2.36 

same 

20 

6.2 

5.75 

0 . 591526 

2.36 

same 

21 

4.2 

4 . 5 

-0.30259 

2.36 

same 

22 

6.2 

5.75 

0 . 855647 

2.36 

same 

II  . 

28.8 

29 

-0.07671 

2.36 

same 

23 

5 

5 . 5 

-0.46546 

2.36 

same 

24 

4.8 

4 

0 . 571887 

2.36 

same 

25 

7 

6 . 5 

1.314684 

2.36 

same 

26 

6.2 

5 . 5 

1 . 118748 

2.36 

same 

27 

4.4 

5 

-0 . 48045 

2.36 

same 

28 

4 . 8 

6.25 

-1 . 53378 

2.36 

same 

29 

5.8 

5.5 

0.684271 

2.36 

same 

30 

6 . 2 

6 

0.407884 

2.36 

same 

31 

6.2 

5.75 

0 . 855647 

2.36 

same 

32 

5 . 2 

6 

-0 . 77583 

2.36 

same 

33 

5.8 

5.75 

0 . 071996 

2.36 

same 

34 

5 . 8 

5.75 

0 . 065007 

2.36 

same 

35 

4 

5 . 5 

-1 .60718 

2.36 

same 

36 

4 . 6 

5 

-0 . 38651 

2.36 

same 

37 

5.4 

5.5 

-0. 10426 

2.36 

same 

III  . 

81.2 

83.5 

-0.28857 

2.36 

same 

Question 

Treatment 

Average 

Contro 1 

Average  t 

tcrit 

Resul t 

38 

2.8 

3.75  -0.83987 

2.36 

same 

39 

5.2 

6  -0.99086 

2.36 

same 

40 

5.4 

5.5  -0. 11728 

2.36 

same 

IV. 

13.4 

15.25  -0.72631 

2.36 

same 

41 

5.2 

6  -1.07344 

2.36 

same 

42 

3.2 

5.25  -1.96662 

2.36 

same 

43 

4 

5.25  -1.22631 

2.36 

same 

44 

4.6 

4  0.470523 

2.45 

same 

45 

5 

5.25  -0.21639 

2 . 36 

same 

V. 

17.4 

21.75  -1.33273 

2.36 

3ame 

46 

5 

5.5  -0 . 40296 

2.36 

same 

47 

4 

5.75  -2.63475 

2.36 

changed 

48 

4 

4.75  -0.58969 

2.36 

same 

49 

4 

5.75  -1.79034 

2.36 

same 

50 

4 . 2 

5.25  -0.891 

2.36 

same 

51 

4.6 

6  -1.7155 

2.36 

same 

52 

4.4 

6  -1.40896 

2.36 

same 

53 

3.2 

5  -1.74561 

2.36 

same 

54 

4.8 

4.75  0.052889 

2.36 

same 

VI  . 

38.2 

48.75  -1.55862 

2.36 

same 

Total 

262 . 2 

283.5  -0.72956 

2 . 36 

same 

335 


Appendix  J.  Comparison  of  Group  Pre-Test  Variance 


Question 

Treatment 

Control 

Pooled 

Variance 

Variance 

Variance 

f 

f CritResult 

1 

0.64 

1.6875 

1 . 088929 

2.636719 

6.59 

same 

2 

0. 16 

0.75 

0 .412857 

4.6875 

6.59 

same 

3 

0. 16 

0.75 

0 .412857 

4.6875 

6.59 

same 

4 

1 . 76 

3 

2 . 291429 

1.704545 

6.59 

same 

5 

0.56 

0.5 

0 . 534286 

1  .  12 

9. 12 

same 

6 

4  .  16 

2.25 

3 . 341429 

1 . 848889 

9. 12 

same 

7 

1 .04 

0.25 

0 . 701429 

4. 16 

9. 12 

same 

8 

2.96 

0.5 

1.905714 

5.92 

9. 12 

same 

9 

3.76 

0.6875 

2.443214 

5.469091 

9. 12 

same 

10 

4.56 

1 . 1875 

3 . 1 14643 

3.84 

9 . 12 

same 

1 1 

1 . 84 

2.888889 

2. 18963 

1 . 570048 

6.94 

same 

12 

1 .36 

0 . 1875 

0.8575 

7.253333 

9. 12 

same 

13 

2.24 

6 

3.493333 

2.678571 

6.94 

same 

14 

0.64 

0.75 

0.687143 

1  .  171875 

6.59 

same 

15 

1 .6 

0.6875 

1 . 208929 

2.327273 

9. 12 

same 

16 

2.56 

3 

2.748571 

1 . 171875 

6 . 59 

same 

17 

1 . 44 

1 . 5 

1 . 465714 

1.041667 

6 . 59 

same 

I  . 

173.76 

204 . 1875 

186.8004 

1  .  175112 

6 . 59 

same 

18 

0.24 

0.25 

0.244286 

1.041667 

6.59 

same 

19 

1.36 

0.25 

0.884286 

5.44 

9. 12 

same 

20 

1.36 

1 . 1875 

1 . 286071 

1  .  145263 

9. 12 

same 

21 

1.76 

2.75 

2. 184286 

1 . 5625 

6 . 59 

same 

22 

0.56 

0 . 6875 

0.614643 

1 . 227679 

6.59 

same 

ERR 

6.59 

ERR 

II  . 

12.56 

18.5 

15. 10571 

1 .47293 

6.59 

same 

23 

2.8 

2.25 

2 . 564286 

1 . 244444 

9. 12 

same 

24 

5.36 

3 

4.348571 

1.786667 

9. 12 

same 

25 

0 

0.75 

0 . 321429 

ERR 

6.59 

ERR 

26 

0.96 

0.75 

0.87 

1.28 

9. 12 

same 

27 

3.44 

3 . 5 

3 . 465714 

1.017442 

6 . 59 

same 

28 

2.96 

0.6875 

1 . 986071 

4.305455 

9. 12 

same 

29 

0.56 

0 . 25 

0 . 427143 

2.24 

9. 12 

same 

30 

0.56 

0.5 

0.534286 

1  .  12 

9 . 12 

same 

31 

0.56 

0 . 6875 

0 .614643 

1 . 227679 

6 . 59 

same 

32 

3.76 

0.5 

2 . 362857 

7.52 

9 . 12 

same 

33 

1.36 

0.6875 

1 .071786 

1.978182 

9. 12 

same 

34 

2.  16 

0. 1875 

1 .314643 

11.52 

9 . 12 

changed 

35 

3.2 

0 . 25 

1 . 935714 

12.8 

9. 12 

changed 

36 

3.04 

1.5 

2.38 

2.026667 

9  .  12 

same 

37 

2.64 

1.25 

2 . 044286 

2.112 

9. 12 

same 

III  . 

185.36 

82.25 

141.17 

2 . 253617 

9 . 12 

same 
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Question 

Variance 

Variance 

Variance 

f 

38 

2 . 96 

2.6875 

2.843214 

1  .  101395 

39 

2.  16 

0.5 

1 .448571 

4.32 

40 

2.64 

0.25 

1 .615714 

10.56 

IV. 

19 . 84 

7 . 1875 

14.4175 

2.760348 

41 

2  .  16 

0 

1 . 234286 

ERR 

42 

2.96 

1 .6875 

2 .414643 

1 . 754074 

43 

2.4 

2. 1875 

2 . 308929 

1 .097143 

44 

4.24 

0 . 666667 

3 . 048889 

6 . 36 

45 

2.8 

3 . 1875 

2 . 966071 

1  .  138393 

V. 

27.04 

19 . 1875 

23 . 67464 

1 .409251 

46 

2.8 

4.25 

3.421429 

1 .517857 

47 

1 . 2 

0.6875 

0 . 980357 

1.745455 

48 

2.4 

5 . 1875 

3 . 594643 

2. 161458 

49 

3.2 

0.6875 

2 . 123214 

4.654545 

50 

3.76 

2. 1875 

3.086071 

1 .718857 

51 

1.84 

1 

1 . 48 

1 .84 

52 

4.64 

0.5 

2 . 865714 

9.28 

53 

3.76 

0 . 5 

2.362857 

7.52 

54 

2.96 

0.6875 

1 . 986071 

4 . 305455 

VI  . 

122. 15 

'M  6875 

101.8146 

1.635615 

Total 

2020. 16 

1726.25 

1894 . 199 

1 . 170259 

* 


f Cr i tResul t 

9.12  same 

9.12  s  ame 

9.12  changed 

9.12  same 

9.12  ERR 

9.12  same 

9.12  same 
19.3  same 

6.59  same 

9.12  same 

6.59  same 

9.12  same 

6.59  same 

9.12  same 

9.12  same 

9.12  same 

9.12  changed 

9.12  same 

9.12  same 

9.12  same 

9.12  same 
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Appendix  Comparison  of  Group  Change  Results 


Question  Treatment  Control 


Average 

Average 

t 

tcrit 

Resul t 

1 

-0.6 

0.5 

-1 . 53292 

2.36 

same 

2 

-0.6 

0 

-0.69007 

2.36 

same 

3 

-1.2 

0.25 

-1.44782 

2.36 

same 

4 

-0.8 

1 

-2 . 25176 

2.36 

same 

5 

-0.4 

-0.5 

0 . 220136 

2.36 

same 

6 

0.4 

-0.75 

1 . 632156 

2.36 

same 

7 

-0.8 

0.5 

-1.54663 

2.36 

same 

8 

-0.2 

-0.5 

0 . 558394 

2.36 

same 

9 

-0.6 

0.5 

-2 . 38463 

2.36 

changed 

10 

0 

0.25 

-0 . 42579 

2.36 

same 

1  1 

-0.6 

-0 . 33333 

-0.51602 

2.45 

same 

12 

-0 . 2 

0.5 

-1 . 59663 

2.36 

same 

13 

0.4 

0 

0 . 325779 

2.45 

same 

14 

-0.4 

0 

-0.98601 

2.36 

same 

15 

-0.6 

0.25 

-0.70956 

2.36 

same 

16 

-0.2 

1.25 

-1 .47159 

2.36 

same 

17 

-0 . 4 

-0.5 

0 . 138494 

2.36 

same 

I  . 

-6.6 

2.75 

-1.71526 

2.36 

same 

18 

-1 

-0.25 

-0.92791 

2.36 

same 

19 

-0.6 

-0.25 

-0.41391 

2.36 

same 

20 

-0.8 

-0.25 

-0.58388 

2.36 

same 

21 

-0.6 

1 

-1 . 47677 

2.36 

same 

22 

-0.8 

0 . 25 

-1.26 

2.36 

same 

II  . 

-3.8 

0.5 

-1  .  13981 

2.36 

same 

23 

-0.6 

-1.75 

1  .  109149 

2.36 

same 

24 

-1 

-0 . 25 

-1 .30189 

2.36 

same 

25 

-0.2 

0 . 25 

-1.6185 

2.36 

same 

26 

-0.8 

0 

-1 .00078 

2.36 

same 

27 

0.2 

0.25 

-0.04121 

2.36 

same 

28 

-1 

0 

-2 . 20479 

2.36 

same 

29 

-0.8 

0.25 

-1 . 27787 

2.36 

same 

30 

-1 

0 

-1 . 57135 

2.36 

same 

31 

-0.8 

-0.25 

-0.8854 

2.36 

same 

32 

-0 . 6 

-0.25 

-0.36476 

2.7? 

same 

33 

0 . 2 

0 . 25 

-0.072 

2.^5 

same 

34 

-0.4 

-0.5 

0.099507 

2.36 

same 

35 

1 

0 

1  .  106728 

2.36 

same 

36 

1 

1 

0 

2.36 

same 

37 

0.8 

1 

-0.29943 

2.36 

same 

III  . 

-4 

0 

-0.65524 

2.36 

same 

Question 

Average 

Average 

t 

ter  i  t 

Resul t 

38 

1 

0 

0.742708 

2.36 

same 

39 

0 . 4 

0.25 

0 .210186 

2.36 

same 

40 

-0.4 

0.25 

-0.64861 

2.36 

same 

IV. 

1 

0.5 

0.224079 

2.36 

same 

41 

0.4 

0.75 

-0 . 7812 

2.36 

same 

42 

0.4 

-0.25 

0.844174 

2.36 

same 

43 

0 . 4 

-0.25 

0.782903 

2.36 

same 

44 

1 

1 

0 

2 . 45 

same 

45 

0 

0.5 

-0.51699 

2.36 

same 

V. 

1 . 2 

0.75 

0 . 188552 

2 . 36 

same 

46 

1 . 2 

1 

0 . 155781 

2.36 

same 

47 

2 

0 

3 . 142697 

2.36 

changed 

48 

1 

0 . 5 

0.404651 

2.36 

same 

49 

1 

-0.5 

1.725898 

2.36 

same 

50 

2 

0 . 5 

1.519944 

2.36 

same 

51 

1.4 

0.25 

1 .611423 

2.36 

same 

52 

-0.4 

-1.75 

1 . 024268 

2.36 

same 

53 

0 . 4 

-0.25 

1.027717 

2.36 

same 

54 

0.8 

-0 . 25 

0.96537 

2.36 

same 

VI  . 

9.4 

-0.5 

1 . 84227 

2.36 

same 

Total 

-2.8 

4 

-0.38654 

2.36 

same 
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Appendix  L.  Comparison  of  Group  Change  Variance 


Question 

Treatment 

Control 

Pooled 

Variance 

Variance 

Variance 

f 

fCritBesult 

1 

1.44 

0.75 

1 . 144286 

1 . 92 

9. 12 

same 

2 

1 . 44 

2 

1.68 

1.388889 

6.59 

same 

3 

3.76 

0. 1875 

2 . 228929 

20.05333 

9. 12 

changed 

4 

1.36 

1 . 5 

1.42 

1  .  102941 

6.59 

same 

5 

0.24 

0.75 

0.458571 

3.125 

6.59 

same 

6 

1 .04 

1 . 1875 

1 . 103214 

1  .  141827 

6.59 

same 

7 

2 . 56 

0.25 

1 . 57 

10.24 

9. 12 

changed 

a 

0.56 

0.75 

0 . 641429 

1 . 339286 

6.59 

same 

9 

0.64 

0.25 

0 . 472857 

2.56 

9. 12 

same 

10 

1.2 

0. 1875 

0 . 766071 

6.4 

9. 12 

same 

1 1 

0 . 64 

0.222222 

0 . 500741 

2.88 

19 . 3 

same 

12 

0.56 

0.25 

0.427143 

2.24 

9. 12 

same 

13 

4 . 24 

0 

2 . 826667 

EBB 

19.3 

EBB 

14 

0.64 

0 

0 . 365714 

EBB 

9. 12 

EBB 

15 

5 . 44 

0. 1875 

3. 188929 

29.01333 

9 . 12 

changed 

16 

1.76 

2.6875 

2 . 1575 

1 . 526989 

6.59 

same 

17 

1 .84 

0.25 

1  .  158571 

7.36 

9. 12 

same 

I  . 

97.04 

24.6875 

66.03179 

3.930734 

9. 12 

same 

18 

2.4 

0. 1875 

1 .451786 

12.8 

9. 12 

changed 

19 

2.64 

0. 1875 

1.588929 

14.08 

9. 12 

changed 

20 

2.56 

1  .  1875 

1.971786 

2. 155789 

9. 12 

same 

21 

3.44 

1.5 

2.608571 

2.293333 

9.12 

same 

22 

2.56 

0 . 1875 

1 . 543214 

13.65333 

9. 12 

changed 

II  . 

52 . 16 

4 . 25 

31 . 62714 

12.27294 

9. 12 

changed 

23 

1.04 

4 . 1875 

2.388929 

4.026442 

6.59 

same 

24 

0.4 

1  .  1875 

0.7375 

2 . 96875 

6.59 

same 

25 

0. 16 

0. 1875 

0 . 171786 

1 .  171875 

6 . 59 

same 

26 

1.36 

1.5 

1.42 

1  .  102941 

6.59 

same 

27 

2.96 

3 . 6875 

3 . 271786 

1 . 245777 

6.59 

same 

28 

0 . 8 

0 

0.457143 

EBB 

9. 12 

EBB 

29 

1 . 36 

1 . 6875 

1 . 500357 

1 . 240809 

6.59 

same 

30 

1 . 2 

0.5 

0.9 

2 . 4 

9. 12 

same 

31 

1 . 36 

0 . 1875 

0 . 8575 

7.253333 

9 . 12 

same 

32 

3 . 44 

0 . 1875 

2.046071 

18.34667 

9.  12 

changed 

33 

1.36 

0.6875 

1 . 071786 

1.978182 

9. 12 

same 

34 

2 . 24 

2 . 25 

2 . 244286 

1.004464 

6.59 

same 

35 

2 . 8 

0.5 

1 .814286 

5 . 6 

9.12 

same 

36 

1.6 

1.5 

1 . 557143 

1 . 066667 

9. 12 

same 

37 

1 . 36 

0 . 5 

0.991429 

2.72 

9. 12 

same 

III  . 

62.8 

109 . 5 

82 .81429 

1 .743631 

6.59 

same 

Question 

Treatment 

Variance 

Contro 1 
Variance 

Pooled 

Variance 

f 

f Cr i tResul t 

38 

4 . 8 

3 

4.028571 

1.6 

9. 12 

same 

39 

1.84 

0 . 1875 

1  .  131786 

9.813333 

9. 12 

changed 

40 

2.64 

1.6875 

2 . 231786 

1 . 564444 

9. 12 

same 

IV. 

12.8 

8.75 

1 1 . 06429 

1 . 462857 

9. 12 

same 

41 

0 . 64 

0 . 1875 

0 . 446071 

3.413333 

9. 12 

same 

42 

1 .04 

1 .6875 

1.3175 

1 .622596 

6.59 

same 

43 

1.04 

2 . 1875 

1 . 531786 

2  .  103365 

6.59 

same 

44 

2 . 8 

2 

2 . 533333 

1 . 4 

19.3 

same 

45 

1 . 2 

3.25 

2.078571 

2 . 708333 

6 . 59 

same 

V. 

7.76 

19. 1875 

12 . 6575 

2 . 472616 

6 . 59 

same 

46 

4  .  16 

3 

3.662857 

1.386667 

9 . 12 

same 

47 

1.2 

0.5 

0.9 

2.4 

9. 12 

same 

48 

2 

5.25 

3.392857 

2 . 625 

6.59 

same 

49 

2 

1 . 25 

1.678571 

1.6 

9. 12 

same 

50 

3.6 

0 . 25 

2 . 164286 

14.4 

9. 12 

changed 

51 

1 . 84 

0 . 1875 

1 . 131786 

9 .813333 

9. 12 

changed 

52 

6 . 24 

0.6875 

3.860357 

9.076364 

9. 12 

same 

53 

1 . 04 

0 . 6875 

0 . 888929 

1 .512727 

9.  12 

same 

54 

2.96 

2. 1875 

2 . 628929 

1.353143 

9. 12 

same 

VI  . 

98.24 

18.75 

64 .  17286 

5.239467 

9. 12 

same 

Total 

918.16 

380.5 

687.7343 

2.413035 

9. 12 

same 
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Aggend 1 x  Statistical  Analysis 


Test  1  -  Equivalency  of  treatment  and  control  groups 

Focus  Question:  Are  the  treatment  and  control  groups  equal 
in  their  responses  to  the  54  questions  on  the  Quality 
Questionnaire,  with  95%  confidence. 

Assumptions:  1)  The  population  distributions  for  the 

responses  are  normally  distributed.  This  will  result  in  the 
mean  of  the  treatment  group  responses  (xbar)  and  the  mean  of 
the  control  group  responses  (ybar)  to  also  be  declared 

normally  distributed  for  each  question.  * 

2)  The  sampling  statistic  xbar  -  ybar  is  normally 
distributed  as  a  linear  combination  of  random  variables, 
with  its  expected  value  being  the  true  difference  in  means. 

3)  The  standard  deviation  of  xbar  equals  the  standard 
deviation  of  ybar,  and  is  unknown.  This  equality  will  be 
tested  in  test  2,  following.  The  pooled  standard  deviation 
will  be  estimated,  and  its  difference  from  the  true  standard 
deviation  accounted  for  using  the  T-test. 

Symbols:  xbar  -  treatment  group  sample  mean 

ybar  -  control  group  sample  mean 

Sx  -  treatment  group  sample  standard  deviation 
Sy  -  control  group  sample  standard  deviation 
m  -  treatment  group  sample  size 
n  -  control  group  sample  size 
Sp  -  pooled  standard  deviation  of  sample 

Sp  =  ( (m  -  1)  *  Sx  »  Sx  +  (n  -  1)  *  Sy  »  Sy)  /  (m  +  n  -  2) 

Test  Statistic: 

t  =  (xbar  -  ybar)  /  (sqrt(l  /  m  +  1  /  n)  *  Sp) ,  with 

m  +  n  -  1  degrees  of  freedom. 

Null  Hypothesis:  Treatment  mean  equals  the  control  mean 

Alternate  Hypothesis:  The  two  means  are  different 

The  test:  Compare  t  to  t  critical  using  a  two-tailed  test 
which  accounts  for  a  difference  in  either  direction.  If  the 
absolute  value  of  t  is  greater  than  t  critical  for  alpha  /  2 
=  .025  and  m  +  n  -  1  degrees  of  freedom,  reject  the  null 
and  conclude  that  the  means  are  different.  If  not,  accept 
the  null  and  conclude  that  the  means  are  equivalent. 
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This  statistical  test  was  performed  for  each  of  the  54 
questions.  The  t  statistic  was  compared  to  t  critical  of 
2.36  for  m  +  n  -  1=8  degrees  of  freedom  at 
alpha  /  2  =  .025.  The  means  were  then  declared  to  be  the 
"same*  or  'changed'.  The  results  of  this  test  for  each 
question  are  tabulated  in  Appendix  I.  Of  the  54  questions, 
all  but  one  were  found  the  same.  This  one  exception  can  be 
attributed  to  the  fact  that  at  the  alpha  level  of  .05,  even 
equal  populations  would  be  expected  to  yield  unequal  samples 
five  percent  of  the  time. 

The  conclusion  of  this  test  is  that  the  treatment  and 
control  groups  are  declared  to  be  equal.  This  is  important 
in  controlling  the  regression  obstacle  to  internal  validity. 
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Test  2  -  Equivalency  of  treatment  and  control  group  variance 
f or  Test  1 . 

Focus  Question:  Are  the  population  variances  of  the 
treatment  and  control  groups  equal  in  their  responses  to  the 
54  questions  on  the  Quality  Questionnaire,  with  95*/. 
conf idence . 


Assumptions:  1)  The  sampling  statistic  Sx  *  Sx  /  (Sy  *  Sy) 
has  an  F  distribution,  with  its  expected  value  being  one  if 
the  populations  have  equal  variance. 


Symbols : 


Sx 

Sy 

m 

n 


treatment  group  sample  standard  deviation 
control  group  sample  standard  deviation 
treatment  group  sample  size 
control  group  sample  size 


Test  Statistic: 


f  =  (Sx  «  Sx)  /  (Sy  *  Sy) ,  or  the  inverse  so  that  the 
ratio  is  greater  than  one. 

Null  Hypothesis:  Treatment  group  and  control  group  response 
variances  are  equal. 

Alternate  Hypothesis:  The  two  variances  are  different 

The  test:  Compare  f  to  f  critical.  If  the  value 
of  f  is  greater  than  f  critical  for  alpha  =  .05,  with  m  and 
n  according  to  the  test  statistic,  reject  the  null  and  conclude 
that  the  variances  are  different.  If  not,  accept  the  null  and 
conclude  that  the  variances  are  equivalent.  This  will 
support  the  assumption  made  in  Test  1. 

Thi3  statistical  test  was  performed  for  each  of  the  54 
questions.  The  f  statistic  was  compared  to  i  critical  of 
9.12  for  m  =  5  and  n  -  4  and  Sx  larger  than  Sy  or  f  critical 
of  6.59  for  Sy  larger  than  Sx  at  alpha  =  .05.  The 
variances  were  then  declared  to  be  the  ’same'  or  "changed". 

The  results  of  this  test  for  each  question  are  tabulated  in 
Appendix  J.  Of  the  54  questions,  all  but  four  were  found  the 
same.  These  four  exceptions  can  be  attributed  to  the  fact 
that  at  the  alpha  level  of  .05,  even  equal  populations  would 
be  expected  to  yield  unequal  samples  five  percent  of  the 
time.  On  two  of  the  questions  an  ERRor  message  resulted. 

This  was  because  one  of  the  variances  was  zero,  resulting  in 
an  undefined  ratio. 

The  conclusion  of  this  test  is  that  the  treatment  and 
control  group  variances  are  declared  to  be  equal.  This  is 
important  in  confirming  the  assumptions  made  in  Test  1. 


m 


> 


* 
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Test  3  -  Significance  of  change  of  treatment  group  with 
respect  to  the  control  group 

Focus  Question:  Are  the  treatment  and  control  groups  equal 
in  the  change  of  their  responses  over  a  6  month  period  to 
the  54  questions  on  the  Quality  Questionnaire,  with  95% 
conf idence . 

Assumptions:  1)  The  population  distributions  for  the 

responses  are  normally  distributed.  The  change  in  response 
for  the  control  group,  being  a  linear  combination  of  the 
pre-test  and  post-test  responses,  will  then  also  be  normally 
distributed.  This  will  result  in  the  mean  of  the  treatment 
group  response  changes  (xbar)  and  the  mean  of  the  control 
group  response  changes  (ybar)  also  being  declared  normally 
distributed  for  each  question. 

2)  The  sampling  statistic  xbar  -  ybar  is  normally 
distributed  as  a  linear  combination  of  random  variables, 
with  its  expected  value  being  the  true  difference  in  means. 

3)  The  standard  deviation  of  xbar  equals  the  standard 
deviation  of  ybar,  and  is  unknown.  This  equality  will  be 
tested  in  test  2,  following.  The  pooled  standard  deviation 
will  be  estimated,  and  its  difference  from  the  true  standard 
deviation  accounted  for  using  the  T-test. 

A  two  sample  paired  T-test  will  be  used.  This  is  possible 
because  the  pre-test  can  be  matched  to  the  post-test  for 
each  individual  who  participated  in  the  quasi-experiment. 

Symbols:  xbar  -  treatment  group  sample  change  mean 

ybar  -  control  group  sample  change  mean 
Sx  -  treatment  group  sample  change  standard 
deviation 

Sy  -  control  group  sample  change  standard 
deviation 

m  -  treatment  group  sample  size 
n  -  control  group  sample  size 

Sp  -  pooled  standard  deviation  of  sample  change 

xbar  =  xbar (post- test)  -  xbar (pre-test) 
ybar  =  ybar (post- test)  -  ybar (pre-test) 

Sp  =  ( (m  -  1)  #  Sx  *  Sx  +  (n  -  1)  *  Sy  *  Sy)  /  (m  +  n  -  2) 
Test  Statistic: 

t  =  (xbar  -  ybar)  /  (sqrt(i  /  m  +  1  /  n)  *  Sp) ,  with 
m  +  n  -  1  degrees  of  freedom. 

Null  Hypothesis:  Treatment  mean  change  equals  the  control 
mean  change 


Alternate  Hypothesis:  The  two  means  are  different 


The  test:  Compare  t  to  t  critical  using  a  two-tailed  test 
which  accounts  for  a  change  in  either  direction.  If  the 

absolute  value  of  t  is  greater  than  t  critical  for  alpha  /  2 

=  .025  and  m  +  n  -  1  degrees  of  freedom,  reject  the  null  and 
conclude  that  the  means  are  different.  If  not,  accept  the 
null  and  conclude  that  the  means  are  equivalent. 

This  statistical  test  was  performed  for  each  of  the  54 

questions.  The  t  statistic  was  compared  to  t  critical  of 

2.36  for  m  +  n  -  1=8  degrees  of  freedom  or  t  critical  of 

2.45  for  m  +  n  -  1  =  7  degrees  of  freedom  as  appropriate  at 

alpha  /  2  =  .025.  The  mean  changes  were  then  declared  to  be  » 

the  ‘same*  or  'changed*.  The  results  of  this  test  for  each 

question  are  tabulated  in  Appendix  K.  Of  the  54  questions, 

all  but  two  were  found  the  same.  These  two  exceptions  can 

be  attributed  to  the  fact  that  at  the  alpha  level  of  .05, 

even  equal  populations  would  be  expected  to  yield  unequal 

samples  five  percent  of  the  time. 

The  conclusion  of  this  test  is  that  the  treatment  and 
control  groups’  changes  in  attitude  toward  quality  are 
declared  to  be  equal.  No  significantly  different  change  was 
observed  in  the  treatment  group  than  in  the  control  group. 


Test  4 


Equivalency  of  treatment  and  control  group  variance 
for  Test  3. 


Focus  Question:  Are  the  population  variances  of  the 
treatment  and  control  groups  equal  in  the  change  in  their 
responses  to  the  54  questions  on  the  Quality  Questionnaire, 
with  95%  confidence. 


Assumptions:  1)  The  sampling  statistic  Sx  *  Sx  /  (Sy  *  Sy) 
has  an  F  distribution,  with  its  expected  value  being  one  if 
the  populations  have  equal  variance. 


Symbols : 


Sx 

Sy 

m 

n 


treatment  group  sample  standard  deviation 
control  group  sample  standard  deviation 
treatment  group  sample  size 
control  group  sample  size 


Test  Statistic: 

f  =  (Sx  *  Sx)  /  (Sy  *  Sy) ,  or  the  inverse  so  that  the 
ratio  is  greater  than  one. 

Null  Hypothesis:  Treatment  group  and  control  group  change 
in  response  variances  are  equal . 

Alternate  Hypothesis:  The  two  variances  are  different 

The  test:  Compare  f  to  f  critical.  If  the  value 
of  f  is  greater  than  f  critical  for  alpha  =  .05,  with  m  and 
n  according  to  the  test  statistic,  reject  the  null  and  conclude 
that  the  variances  are  different.  If  not,  accept  the  null  and 
conclude  that  the  variances  are  equivalent.  This  will 
support  the  assumption  made  in  Test  3. 

This  statistical  test  was  performed  for  each  of  the  54 
questions.  The  f  statistic  was  compared  to  f  critical  of 
9.12  for  m  =  5  and  n  =  4  and  Sx  larger  than  Sy ,  f  critical 
of  6.59  for  Sy  larger  than  Sx ,  or  f  critical  of  19.3  for  m  = 

5  and  n  =  3  and  Sx  larger  than  Sy  at  alpha  =  .05.  The 
variances  were  then  declared  to  be  the  'same'  or  'changed’. 

The  results  of  this  test  for  each  question  are  tabulated  in 
Appendix  L.  Of  the  54  questions,  all  but  eleven  were  found 
the  same.  These  eleven  exceptions  are  more  than  can  be 
attributed  to  the  fact  that  at  the  alpha  level  of  .05,  even 
equal  populations  would  be  expected  to  yield  unequal  samples 
five  percent  of  the  time.  However,  this  inconsistency  is 
trivialized  by  the  fact  that  no  significant  differences  were 
found  in  the  means.  Had  there  been  significant  findings, 
the  difference  in  variances  may  have  prompted  non-parametric 
testing  of  the  means,  which  do  not  require  the  assumption  of 


equal  variances.  On  three  of  the  questions  an  EBRor  message 
resulted.  This  was  because  one  of  the  variances  was  zero, 
resulting  in  an  undefined  ratio. 

The  conclusion  of  this  test  is  that  the  treatment  and 
control  group  variances  may  or  may  not  be  equal .  This  is 
important  in  confirming  the  assumptions  made  in  Test  3. 
However,  since  Test  3  produced  no  significant  differences  in 
means,  the  question  of  equality  of  variances  will  not  be 
pursued  further. 
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